| RotoGuru Basketball Forum View the Forum Registry | Self-edit this thread |
|
| 0 |
Subject: TSN Price Change formula Posted by: Guru - [330592710] Mon, Nov 03, 2003, 12:41 What is the underlying TSN price change formula? This discussion has started in the Arroyo thread, but really should command a separate thread, if for no other reason than that we'll never find it in the archives if it's listed under an "Arroyo" title. I did solve the formula for baseball back in 1999. Here is the report: link, Since then, there have been some changes, but as each was introduced, it has generally been diagnosed as an incremental adjustment on top of the original formula. 1. Prices became daily, rather than weekly. The same general formula was used, with only the general sensitivity factor(s) changed. Over the years since daily repricing has been the standard, there have been tweaks to the sensitivity, ratcheting up or down. Usually, this factor remains constant throughout a season, although there was at least one instance where this was clearly not true. 2. Gravity was added. This seems to be a simple deduction for players that qualify. The unknown details are the criteria for qualifications for gravity, and whether "partial gravity" exists for those who are on the cusp. 3. A dampening factor was added, which has the effect of limiting the gain or loss for the most extreme daily changes. When this was first implemented and studied, the general assessment was that the sensitivity factors were applied to the square roots of the buys and sells, rather than the raw numbers. I think these were the signficant changes. I've seen nothing that causes me to suspect that the underlying formula isn't essentially the same as in 1999 (other than the changes noted above). A consequence of this is that pure ownership data - even if totally accurate - is not sufficient to fully reverse engineer the formula. You need to have the underlying buys and sells, since the sensitivity of a buy tends to be less than the sensitivity of a sell, especially when new team formation is still occurring. I'm sure there are other threads (perhaps more likely in the baseball forum) dealing with past attempts to work out the details. I'll leave it to others if they want to ferret any of them out. | ||||||
| 49 | Sludge Sustainer ID: 25919714 Wed, Nov 05, 2003, 15:26 |
Another function that has the same behavior is arctan. That and the logistic function would have been my first choices if I were seeking to dampen the extreme price changes without simultaneously dampening the moderate price changes. Looking at fitting that instead of the logistic curve, it's virtually identical except it would allow for a maximum loss/gain of right at 200. Price Change = 143.51*arctan(9.19*x), where x is as above. Note that the extreme is when every buy is for one player and he has no sells. That makes the "x" for that player log(1+1) - log(1) = 0.0.693. Plug that into the price change equation, and you get 203. 200K is an awfully nice round number, don't ya think? As Dana Carvey once said (many times) while dressed in drag, "How convenient." | |||||
| 50 | Sludge Sustainer ID: 25919714 Wed, Nov 05, 2003, 15:44 |
More picture goodness: This shows the curves projected to the absolute extremes as described above. I wouldn't be doing my job if I didn't point out that this is extrapolation, and extrapolation is quite often pure crap. I should also be taking points off of myself for not properly labeling my graphs. I'll fix that. | |||||
| 51 | RecycledSpinalFluid Dude ID: 127421915 Wed, Nov 05, 2003, 15:48 | LOL: "extrapolation is quite often pure crap" | |||||
| 52 | Guru ID: 330592710 Wed, Nov 05, 2003, 16:03 | Would that be better termed "excrapolation"? | |||||
| 53 | Blooki@Work ID: 24920611 Wed, Nov 05, 2003, 16:36 |
Guru, Grant Hill (02-03) There's one for you. I'm not sure what to conclude from it. I'm sure those days where Hill defied gravity are not due to trading activity regarding Hill. This would lead me to believe that trading activity or ownership of players other than Grant Hill would somehow affect the "in/out of gravity" calculation for Hill. I don't have any more significant data on this, but I would appreciate it if those who are running tests could be on the lookout for anomalies such as this one. I remember mentioning this last year, but at the time nobody seemed interested in cracking intricacies of the pricing algorithm and I'm glad this discussion has piqued everyone's interest now. | |||||
| 54 | Guru ID: 330592710 Wed, Nov 05, 2003, 16:44 | Yeah, I think there were several last year who seemed to shift in and out of gravity with no obvious reason. although I don't think anyone ever tried to run ownership samples on those players to be able to assess the underlying data. | |||||
| 55 | Astade ID: 214361313 Thu, Nov 06, 2003, 23:55 |
Hey Guys, I was hoping to investigate the price change formula myself, but this last week I've been busy with work. My question to the general audience who have been following this (especially Sludge), weren't there 2 different formulas for price changes? One for Gainers and one for Losers? Don't get me wrong, you have done a great job fitting the current data available, but from all the background research/releases (from TSN), wasn't it decided that %Sells would have a different weight than %Buys?...Yet, Sludge you seem to combine the two into one universal equation? If I'm completely off-base, please correct me quickly, so I can self-edit (and delete my post) ;) | |||||
| 56 | Sludge Sustainer ID: 24981818 Fri, Nov 07, 2003, 00:52 |
Astade - Ummmm, maybe. The data don't seem to support that hypothesis, however. It might explain the "shoelaces" as Guru has called them, but I doubt it. Even if it does, how much will it kill you to be off by at most 10K? (With the usual caveat that we haven't checked the model on the extreme price changes yet.) I will try it with two different scale parameters for the two tomorrow, though, and see if that improves the fit. I don't haunt the hoops forum, but I thought that the reason that people assign a different weight to buys versus sells is because there will always be more buys than sells, so that when they are scaled by dividing by the total buys and sells (respectively), each buy has a weight of 1/Total Buys and each sell has a weight of 1/Total Sells (paradoxically, this actually removes the inordinate weighting that would be assigned if we were to just use the raw number of buys/sells). When building the model, I operated under the (possibly mistaken) assumption that it would be silly to say that a player that gets X% of the total buys and X% of the total sells would move in price at all. In that case, either of the models above would predict a price change of 0. If each variable is assigned a different weight, then this won't be the case and, as you can see from the data, isn't the case. | |||||
| 57 | Sludge Sustainer ID: 24981818 Fri, Nov 07, 2003, 00:55 | Err... umm... that last bit should read, "...and, as you can see from the data, is the case. | |||||
| 58 | Tortfeasor Sustainer ID: 458192922 Fri, Nov 07, 2003, 01:14 |
Blooki- I'm no mathemetician/statistician, but your Grant Hill example makes me wonder: perhaps there are certain minimum values accorded to a player? I know the minimum is supposedly 500K, but why couldn't they set the bar higher for some players? Just a thought, may be way off base here. | |||||
| 59 | Guru ID: 330592710 Fri, Nov 07, 2003, 08:47 |
Astade - previous research showed that the weighting of a buy was different than the weighting of a sell. But that is because there are a different number of total buys than total sells. 100 buys might represent 5% of the total buys, while 100 sells might simultaneously represent 6% of the total sells. Buy first converting buys and sells to percentages, the need for different weightings has been taken into consideration. In past incarnations, we sometimes factored down the buys by a constant percentage so that total adjusted buys were equal to total sells. Then we subtracted sells from factored buys. Ultimately, we also multiplied by a sensitivity factor that had the total number of sell in the denominator. If your rearrange the terms, that is exactly what Sludge's formula does as well. So this is not inconsistent with the previous findings. | |||||
| 60 | Blooki@Work ID: 24920611 Fri, Nov 07, 2003, 09:50 |
Tortfeasor, That would be a possible explanation for why Hill stops gravitating at $4,200,000, but if you look earlier, he occasionally stops and resumes gravitating at other points in the season for absolutely no good reason. I find it extremely unlikely that he's being traded at these times. | |||||
| 61 | Tortfeasor Sustainer ID: 37948287 Fri, Nov 07, 2003, 14:24 |
Blooki- Right. But it could be that some teams that are inactive had him, checked them once, made one or two sells, thus taking him out of gravity. If I remember correctly, gravity applies only when a player is not being bought or sold at all. So if you sold him, that could take him out of gravity, right? I never understood whether sells could take a player out of gravity or not. Just a guess. | |||||
| 62 | Guru ID: 330592710 Fri, Nov 07, 2003, 14:52 |
The recent data does indicate that several buys can take a player out of gravity - although a single buy doesn't seem to be enough. I haven't found the empirical evidence to suggest that sells do or don't have a similar impact. | |||||
| 63 | Deadeyes ID: 50104029 Thu, Nov 13, 2003, 19:00 | INteresting stuff you got going on here. If you give me the data i can probably solve it using matlab. | |||||
| 64 | Pacers Rule ID: 4095190 Thu, Nov 13, 2003, 19:08 | Deadeyes, welcome to the forum of the big boys. You have to behave over here. If you don't, just find a post by Jerry Lewis to see what will happen to you... | |||||
| 65 | Perm Dude Dude ID: 30792616 Thu, Nov 13, 2003, 19:11 |
hola, deadeyes. Was hoping you'd make it to the forum. pd | |||||
| 66 | Sludge Leader ID: 24981818 Thu, Nov 13, 2003, 23:08 |
It's not solved already? Hmph. I thought I did a pretty damn good job. The data's linked in #12. There's only two day's worth there. I'm still waiting for RSF to automate the whole thing. :) | |||||
| 67 | Sludge Leader ID: 24981818 Thu, Nov 13, 2003, 23:25 |
By the way, with more data (I don't think the data set we have now is rich enough, which is why I really didn't spend much time on it), the way I would approach solving the gravity problem is using discriminant analysis. For those that don't know what it is, I'll just quote MINITAB's help file:
Seems like the right hammer for this nail to me. So basically, all we would have to do is code each player for each day as subject to gravity or not (there will surely be some errors inducing a bit of random variation, but that's not a problem), and my first guess as discriminants would be indicator functions of raw buys, raw sells, and ownership. (My second guess would be percentages of the above.) Another option would be logistic regression since the dependent variable is dichotomous. A nice graphical technique would be to plot Buys vs. Sells vs. Ownership (a three dimensional plot) coloring the gravity points red and the rest black. Figure out what surface separates the points, and you have your rule. My guess is that it might be something as simple as a plane. | |||||
| 68 | RecycledSpinalFluid Dude ID: 204401122 Thu, Nov 13, 2003, 23:27 | Sludge, the DB (uncompressed is 50+ MB) but I will make the share I showed you available again (Now that I've recovered from router and server issues). | |||||
| 69 | Sludge Leader ID: 24981818 Thu, Nov 13, 2003, 23:34 |
Oh man, you want me to go through all of that again? :) That code I sent you only goes so far (just listing players bought and players sold... it doesn't even aggregate them yet, although that's a 1 minute coding job to get it to do that since the procedures are already built into R). The real pain is matching up the price changes with the players bought and sold. I'll do it if, uhhh, 5 people donate 5 bucks each to Guru. Yeah, that's the ticket. | |||||
| 70 | RecycledSpinalFluid Dude ID: 204401122 Thu, Nov 13, 2003, 23:38 | I can do something about the price change thing that should make it easier for you. You mind sending me the URL for the price change thing (I used to have it, but now it is lost in the maze of links on my network). | |||||
| 71 | Sludge Leader ID: 24981818 Thu, Nov 13, 2003, 23:39 | Guru can probably beat me to it, as I have it in my Outlook at the office and can't get to it until tomorrow morning. But if he doesn't, I'll send it over to you. | |||||
| 72 | Deadeyes ID: 50104029 Thu, Nov 13, 2003, 23:59 | Just got back from the bars. Who is jerry lewis? what happened to him. I have a short fuse. Not sure if i can last with the big boyz. But i can probably try to solve the equation for the price movers when i have time. Good night. Is this more popular than the chat rooms in the other forum on TSN? | |||||
| 73 | Guru ID: 330592710 Fri, Nov 14, 2003, 08:49 | TSN Ultimate Hoops daily prices. (Let me know if you need the codes to match player IDs into names.) | |||||
| 74 | Guru ID: 330592710 Fri, Nov 14, 2003, 08:54 |
This file has the player IDs in it. Columns 4-7 are the ID#, and the name follows shortly thereafter. | |||||
| 75 | Deadeyes ID: 198319 Fri, Nov 14, 2003, 09:28 | Does it have anything to do with points scored by the player or just ownership buy/sell? Do we know this? From the graph it doesnt seem to be just by % ownership. Looks like another variable is involved. | |||||
| 76 | Guru ID: 330592710 Fri, Nov 14, 2003, 09:37 | Points scored are not a factor. | |||||
| 77 | Sludge Leader ID: 511027148 Fri, Nov 14, 2003, 09:38 |
Entirely possible, deadeyes, but I would guess not. If there were another variable that is used to compute price changes, it would have to have a very small impact. The two models above are off about 10% of the time by 10K in either direction (and nothing more than that in the sample that I have), so that means that any other variables included would have a very low impact on the price change and, to my way of thinking, why would they bother including them then? (All of this begs the question, what's the correlation between points scored and ln(%Buys+1)-ln(%Sells+1)?) My guess, though, is that we've got the variables, but we might not have the exact form that they are used. Why did I use ln(%Buys+1)? Why not log base 10? Why not ln(%Buys+C), where C is something other than 1? | |||||
| 78 | Guru ID: 330592710 Fri, Nov 14, 2003, 09:39 | ...and I think the graph looks pretty tight. Why do you think the graph shows evidence of another factor? | |||||
| 79 | Sludge Leader ID: 511027148 Fri, Nov 14, 2003, 09:45 |
Actually, it doesn't matter which base you use for the log: ln-base-N(x) = ln(x)/ln(N) | |||||
| 80 | Sludge Leader ID: 511027148 Fri, Nov 14, 2003, 09:47 |
Oops... that should be log-base-N(x) = ln(x)/ln(N) So used to just typing "ln" when I'm writing about logarithms. | |||||
| 81 | Smartone @work ID: 22815228 Fri, Nov 14, 2003, 13:10 |
a little OT, but I was wondering if you could help me, as I am interested to build a "next-generation" simulation tool based on historical data. In order to do so I need to extract information from the web. Unfortuately, I am using Office 97 (on Windows XP) and I couldn't, so far, understand how "web queries" work. Can anyone give me a little help here? Unfortunately, the only "programing language" I know is Excel and a little Access. My goal is to have an input database of all of last year's (perhaps this yaer too) games and the TSNP of each player in these games (Minutes Played can also be useful). | |||||
| 82 | RecycledSpinalFluid Dude ID: 510441411 Fri, Nov 14, 2003, 13:16 | Smartone, actually, if you are using Office 97, you don't need to use web queries, just open the URL as you would a file. | |||||
| 83 | Guru ID: 330592710 Fri, Nov 14, 2003, 13:34 | smartone, not quite sure what data detail, you need, but I have an Excel spreadsheet that has minutes played for each player for each game, as well as TSNP for each game. I could provide these for several years as well. Shoot me an email if you're interested. | |||||
| 84 | Smartone @work ID: 22815228 Fri, Nov 14, 2003, 13:55 |
thanks a lot, Guru! I think that 2002/03 data as well as this year's will be great | |||||
| 85 | Deadeyes ID: 50104029 Sun, Nov 16, 2003, 20:06 |
Hey when you used excel did you use the rounding up or down function? i wonder if you did if the points would match up.. I believe this si the function you need to use: ROUND(number,num_digits) i believe the number can be the equation. .try it and see if the points match and let me know. | |||||
| 86 | Guru ID: 330592710 Sun, Nov 16, 2003, 20:16 |
Yes, I used the excel rounding function. I could not get an exact match, but I suspect the rounding approach is the culprit, as most of tracking errors were on values which were close to the middle of each rounding range. The TSN program may round at several intermediate steps. Or perhaps some of the underlying factors are just slightly off. Or both. | |||||
| 87 | Pacers Rule ID: 4095190 Sun, Nov 16, 2003, 23:24 | Something is rotten in Denmark. Looking at a comparison between change in ownership and price changes the other evening (with the naked eye, no stats program needed) found that proportion of player sells to total sells cannot be the only factor in price changes. I saw that a player who had a delta of around 600 didn't really change in price that much more than a player with a delta of 100. Seems like with a 6:1 ratio of sells, there would have been a bigger difference. Sorry I don't have the exact examples. I didn't write them down but thought that later I might remark about in case anyone cared to check into it. I wondered if it might also have something to do with total buys/sells in relation to total ownership, as the 100 delta was on a lot less owned player than the 600 or so. Each delta represented a similar percentage respective to each player, if that made any sense. like each player had 20% of it's owners selling for example. | |||||
| 88 | Guru ID: 330592710 Sun, Nov 16, 2003, 23:31 | Your going to need to point out that example. I haven't seen anything like that. | |||||
| 89 | penngray Sustainer ID: 423241723 Mon, Nov 17, 2003, 11:49 |
smartone, Re #81 if you are considering doing any serious web based parsing look into learn vbscript and/or even better Python or PHP (web development tools). Handling of web page source is much more robust once you learn any of these. Plus I really think once you learn them you will be able to generate your own very functional web pages for simulations. Just ask RSF how much simpler things are with scripts ;) | |||||
| 90 | Sore Thumb ID: 571049813 Wed, Nov 19, 2003, 15:15 |
This is off the topic a little but I was wondering if there is anyway in determining how many different people use the hoops forum. I would like to see the ratio of guru users compared to the 5,000 people in the TSN game. Kind of see how big of an influence these boards might have. There is no way in knowing how many view but dont post but a round about figure could be interesting. Thanks Guru. | |||||
| 91 | Guru ID: 330592710 Wed, Nov 19, 2003, 15:46 |
There is no accurate way that I can monitor this, because a lot of people come from multiple points of entry (home, work, school, library, etc.) There is also no easy way to know how many distinct managers are represented in the 5000+ teams - although perhaps this is something that RSF could track in his sampling, by keeping track of manager name (or number) in addition to team number. It would be interesting, though. I'll set up a separate thread to start an informal survey. | |||||
| 92 | smartone Donor ID: 29452720 Sat, Nov 22, 2003, 10:59 | can anyone check why Chris Bosh continues to gravitate(?) while his ownership level is over 2% and people continue to buy him? | |||||
| 93 | Guru ID: 330592710 Sat, Nov 22, 2003, 11:28 |
Just glanced at Bosh's ownership, and he does looks like an anomaly. His ownership on Nov.20 was 140 teams, more than 2.6% of rosters, which seems to be well above the gravity threshold that was in place earlier in the month. He also had 11 buys on 11/20, which should have kicked him out. It might be time to get some complete details and rerun the numbers to see whether there is a plausible explanation that is consistent with the prior data. | |||||
| 94 | RecycledSpinalFluid Dude ID: 510441411 Sat, Nov 22, 2003, 15:41 | The gravity cutoff looks like its at about 3%, which would include Bosh. Haven't looked at past numbers, but 2.8-2.9ish seems to be a number that is sticking in my head from previous looks. | |||||
| 95 | Guru ID: 330592710 Sat, Nov 22, 2003, 16:44 | RSF - based on the earlier analysis in [26] and [28], the gravity cutoff was around 2%. | |||||
| 96 | JCS Sustainer ID: 20102934 Fri, Dec 05, 2003, 11:50 | Hmmm.. let's look at the latest ownership numbers. Murray gets a -344 delta and loses 80k, and Billups gets a -349 delta yet loses 100k. This may be an evidence that the player's price is a factor in his price change. | |||||
| 97 | Guru ID: 330592710 Fri, Dec 05, 2003, 13:11 |
It may also be a consequence of differences in the buy/sell distribution of those net changes. For example, Billups may have had 349 sells and no buys, while Murray may have had 364 sells and 20 buys. That might cause a slight difference in the formula change. Rounding could also account for part of the difference. | |||||
| 98 | Pacers Rule ID: 910311210 Fri, Dec 05, 2003, 15:00 |
RE #87 I haven't confirmed this for a fact, but the thought crossed my mind that I could have been looking at the current price moves while looking at the previous day's ownership numbers, which would explain why there wasn't a match! I probably forgot that the ownership numbers don't come out until mid-day the next day. | |||||
| Rate this thread: | If you wish, you may rate this thread on scale of 1-5. Ratings should indicate how valuable or interesting you believe this thread would be to other users of this forum. A '5' means that this thread is a 'must read'. A '1' means that this is a complete waste of time. If you have previously rated this thread, rating it again will delete your previous rating. If you do not want to rate this thread, but want to see how others have rated it, then click the button without entering a rating, or else click here. | ||||||
| |||||||
Post a reply to this message: (But first, how about checking out this sponsor?) | |||||||