OOTP Developments Forums - View Single Post

RonCo · 07-13-2008, 11:40 AM

To study the effect of the AB league totals modifier, I took a vanilla league of 8 teams and ran a total of four seasons. The first two seasons I set the AB league total to be very high (1,000,000) without changing any other total value. After two seasons I set the league totals very low (90,000):

Default League Totals:

AB: 167353
H: 44522
2B: 8919
3B: 898
HR: 5451
BB: 16222
HBP: 1850
K: 31828
BABIP: .296

Here are the results.

Hitting:

Code:

	R/G	G	AB	H	2B	3B	HR	BB	SO	SB	AVG	OBP	SLG	OPS
2008	36.2	1296	54928	22576	4874	580	3900	40852	30241	1159	.411	.681	.734	1.415
2009	34.9	1296	54350	21946	4641	548	3683	40501	30133	1103	.404	.676	.713	1.389
2010	2.1	1296	43199	6608	1646	166	876	2323	3864	282	.153	.200	.260	.459
2011	2.1	1296	42813	6657	1649	154	854	2245	3675	295	.155	.201	.261	.462

Pitching:

Code:

	R/G	G	CG	SHO	SV	ERA	IP	HA	HRA	BB	K	WHIP	OAVG	BABIP
2008	36.2	1296	26	0	55	36.55	11329.1	22576	3900	40852	30241	5.600	0.411	.898
2009	34.9	1296	24	0	75	35.09	11366	21946	3683	40501	30133	5.490	0.404	.889
2010	2.1	1296	408	230	361	1.73	12410.2	6608	876	2323	3864	.720	0.153	.149
2011	2.1	1296	444	268	335	1.71	12270.2	6657	854	2245	3675	.730	0.155	.152

Now let's take a look at some specific things we can learn from the results.

1) The chasm between 2008-9 (AB Totals High) and 2010-11 (AB Total Low) show us how radical changes in the AB totals effect every league total category. Mechanically, this is because the League total ratios are used as League Average values in the Bill James Log5 (or see Odds Ratio in math/stats books, they are essentially the same).

Given this, raising the AB League Total makes the expected average at bat result in low offense (the expected average hitter in this league is poor, the expected average pitcher in this league is good). Since our actual batters have higher ratings than this, it makes our normally average major league hitter look like Sammy Sosa in his prime, and he achieves great results. Make sense--by setting the league totals with AB very high, we've told the game that we're playing in a High School level league, and placed a major league hitter into it. Similarly, if we make league totals AB drop, we tell the game that the average hitter is really good. So our normal major league player will not perform as well.

It's a little more complex than this, of course. This is because of the nature of league totals. This leads us to the question ...

2) Just what _are_ league totals representing?

This is a great question. Some folks think they are a target that the league is shooting for. This is wrong. Some folks think they are just mechanical bars you use to influence outcomes. While this is technically correct, it is not complete enough for the question. I state above that the league totals represent the expected result of the average pitcher/hitter matchup. This is correct, but not deep enough.

From a modeling perspective, the result of a real-life AB is molded by a collection of things: 1) ability of the hitter, 2) ability of the pitcher, 3) ballpark, 4) defense behind the pitcher, 5) "context of the league." OOTP has ratings that model items (1), (2), (3), and (4). The game has no immediate way to determine (5).

Item (5)--league context--is what League Totals are here to model.

So, then ...

3) What is league context in OOTP terms?

"Context" is a catch-all that includes lots of things. It's easiest to think of as a definition of the average result of the pairing between an average pitcher and average hitter, but that's an oversimplification and (despite the fact that I say it all the time) is "wrong." It's really a way to define the effect of rules differences, or training differences, or whatnot. For example, a league total change is required to make the league behave properly to the late 60s process of lowering the mound and reducing the strike zone area. A league total change is required to reflect the rule change about marring the ball, or to reflect lighter/heavier bats. Play a league on Mars, and a league total change is required.

This is a real issue from a modeling perspective. League totals are attempting to do two things: (1) Define the quality of players in the league, and (2) define the special environmental rules in effect.

You can tell this because you have to go to league totals in order to influence either historical baseball era settings, or to influence minor league performance in any league. This is a flaw, actually. I'm not educated enough on the actual algorithms to know what to do to correct it, but it seems intuitive that Markus could and should get "league average" by doing a true average of the players in the league at any time--this would be calculated in the background and perhaps displayed in a set-up screen. This would remove that aspect of the model fromthe league totals, leaving only an "Era/Environmental adjustment" setting. I think this would make the whole thing less confusing.

4) What have we done to the model to create these stats?

Since the OOTP League Totals model includes "expected quality of the players in the league" it is assuming some things about the ratings of players in the league. In other words, if the average quality (ratinngs) of the players in the league are in the ballpark of the league totals, you'll get those results. What we've done, though, is tell the game engine that we've got a league of a level of quality that is far different from the actual players in the league. This means that almost every AB is being made by a player that the game engine views as being an outlier. If the game engine thinks every hitter is effectively made into a Steve Jeltz, or a Babe Ruth, then you can see that things will get out of whack quickly. So essentially we are either lying to the game engine with league totals by telling it that the average player is far out of whack with reality, or by entering these skewed League Totals we're telling the game engine tha the rules/environment we've selected to play in is very different from "real baseball."

All that said, there are still things to learn from this set of league data...

5) When you adjust the AB Totals, you can see how the game has implemented the DIPS theory.

DIPS says that pitchers control only strikeouts, walks, and HR. So it's interesting to look at these numbers as we adjust AB. When we have High AB League Totals, we find that strikeouts soar. Remember that every result is influenced by pitcher and hitter. So this is because the "league average pitcher" is very poor, so a major league pitcher will strike out a ton of hitters in this environment. Similarly, walks soar because the league average hitter will almost never walk, and the average major league hitter and pitcher walk and give up more walks. Home runs also soar, for the same reason.

Of more interest, though, is BABIP.

BABIP in this environment is an astronomical .898/.890. So almost every result is either a hit or a strikeout.

Similarly, when we lower the AB League total we find the counter to be true--BABIP drops to a paltry .149/.152.

What the heck? We didn't change the BABIP setting at all. How can this be?

6) What's going on with BABIP?

Time for a caveat. I really know nothing about how the game engine does its thing. Markus has described it before, but in the end only he knows the truth of the engine. But here's my theory, the results engine is not as robust as it needs to be because it determines results in the wrong order.

I've said this lots of times, and shown thought-logic to defend my position. But the data above, in my mind, confirms this beyond any shadow of a doubt. Two other things could be aggravating this -- (1) the that BABIP rating could be mapped to values that are too high, but I would guess that's not the case as Markus is ususally pretty diligent about those kinds of things and that would be a moderately big and obvious miss. (2) Markus could still be using the Contact rating to determine events, but I don't think he is.

Let's look at some additional data:

Code:

League Batting Stats												
	AB/G	H/AB	2B/AB	3B/AB	HR/AB	BB/AB	SO/AB	BABIP		2B/H	3B/H	HR/H
2008	42.4	.411	.089	.011	.071	.744	.551	.898		.216	.026	.173
2009	41.9	.404	.085	.010	.068	.745	.554	.889		.211	.025	.168
2010	33.3	.153	.038	.004	.020	.054	.089	.149		.249	.025	.133
2011	33.0	.155	.039	.004	.020	.052	.086	.152		.248	.023	.128
												
League Pitching Stats												
	H/9	HR/9	BB/9	K/9	K/BB							
2008	17.9	3.1	32.5	24.0	.740							
2009	17.4	2.9	32.1	23.9	.744							
2010	4.8	0.6	1.7	2.8	1.663							
2011	4.9	0.6	1.6	2.7	1.637

In 2008/2009 you see that everything happens more often--pitchers and hitters walk more and strike out lots more (24 Ks/game mean only three outs happen on Balls in Play). Teams hit 3 HR/game. We constrain K, HR, and BB (which I think get calculated first), so all the game engine has to work with are balls in play. You can see that the game engine is doing its best in the 2008/2009 ears to make hitters really good. So either the game is trying to control batting average or BABIP. Since BABIP is fluctuating oddly, I assume it's attempting to control average--which makes sense given that the game is determining if a result is a hit too early in the process.

Note, also, the three columns to the right (2B/H, 3B/H, and HR/H). Note espeically that our 2B/H and HR/H ratios behave in inverse when the totals flip-flop in dramatic fashion. By this I mean the numbers of 2B/H increase while the numbers of HR/AB decrease, and visa versa. I think this is an important indicator of something, and could speculate on it ... but only Markus would know for sure.

7) Last thoughts

This exercise is so interesting because it's stretching the game engine a couple ways--it's saying hitters are better or worse in the same breath (increasing BB, SO, HR all at the same time), and the same for pitchers (increasing BB, SO, and HR). It gives you lots to think about, and shows exactly how complex the game engine really is. It also exposes a few places where it struggles to provide results due to the model's abstraction being made at too high of a level to properly deal with more detailed issues. This is the case for pretty much all baseball simulations (think of the major abstractions in our old dice or spinner games), but it drives a lot of confusion when you get into the details.

I recommend that sometime in the future Markus:

1) Calculate the real league average player rating and use that in the definition of League Totals (I'm sure this will be complex, and cause further issues with the historical/fictional crowds)
2) Provide a "Era Adjustments" screen rather than league totals
3) Add the use of GB/FB/LD data to the early stages of the results engine, and determine defense, hit, and HR events at the proper locations (again, I'm sure this is complex).

07-13-2008, 11:40 AM	#12 (permalink)
RonCo Hall Of Famer Join Date: Aug 2003 Posts: 6,218 Thanked 3x in 2 posts	To study the effect of the AB league totals modifier, I took a vanilla league of 8 teams and ran a total of four seasons. The first two seasons I set the AB league total to be very high (1,000,000) without changing any other total value. After two seasons I set the league totals very low (90,000): Default League Totals: AB: 167353 H: 44522 2B: 8919 3B: 898 HR: 5451 BB: 16222 HBP: 1850 K: 31828 BABIP: .296 Here are the results. Hitting: Code: R/G G AB H 2B 3B HR BB SO SB AVG OBP SLG OPS 2008 36.2 1296 54928 22576 4874 580 3900 40852 30241 1159 .411 .681 .734 1.415 2009 34.9 1296 54350 21946 4641 548 3683 40501 30133 1103 .404 .676 .713 1.389 2010 2.1 1296 43199 6608 1646 166 876 2323 3864 282 .153 .200 .260 .459 2011 2.1 1296 42813 6657 1649 154 854 2245 3675 295 .155 .201 .261 .462 Pitching: Code: R/G G CG SHO SV ERA IP HA HRA BB K WHIP OAVG BABIP 2008 36.2 1296 26 0 55 36.55 11329.1 22576 3900 40852 30241 5.600 0.411 .898 2009 34.9 1296 24 0 75 35.09 11366 21946 3683 40501 30133 5.490 0.404 .889 2010 2.1 1296 408 230 361 1.73 12410.2 6608 876 2323 3864 .720 0.153 .149 2011 2.1 1296 444 268 335 1.71 12270.2 6657 854 2245 3675 .730 0.155 .152 Now let's take a look at some specific things we can learn from the results. 1) The chasm between 2008-9 (AB Totals High) and 2010-11 (AB Total Low) show us how radical changes in the AB totals effect every league total category. Mechanically, this is because the League total ratios are used as League Average values in the Bill James Log5 (or see Odds Ratio in math/stats books, they are essentially the same). Given this, raising the AB League Total makes the expected average at bat result in low offense (the expected average hitter in this league is poor, the expected average pitcher in this league is good). Since our actual batters have higher ratings than this, it makes our normally average major league hitter look like Sammy Sosa in his prime, and he achieves great results. Make sense--by setting the league totals with AB very high, we've told the game that we're playing in a High School level league, and placed a major league hitter into it. Similarly, if we make league totals AB drop, we tell the game that the average hitter is really good. So our normal major league player will not perform as well. It's a little more complex than this, of course. This is because of the nature of league totals. This leads us to the question ... 2) Just what _are_ league totals representing? This is a great question. Some folks think they are a target that the league is shooting for. This is wrong. Some folks think they are just mechanical bars you use to influence outcomes. While this is technically correct, it is not complete enough for the question. I state above that the league totals represent the expected result of the average pitcher/hitter matchup. This is correct, but not deep enough. From a modeling perspective, the result of a real-life AB is molded by a collection of things: 1) ability of the hitter, 2) ability of the pitcher, 3) ballpark, 4) defense behind the pitcher, 5) "context of the league." OOTP has ratings that model items (1), (2), (3), and (4). The game has no immediate way to determine (5). Item (5)--league context--is what League Totals are here to model. So, then ... 3) What is league context in OOTP terms? "Context" is a catch-all that includes lots of things. It's easiest to think of as a definition of the average result of the pairing between an average pitcher and average hitter, but that's an oversimplification and (despite the fact that I say it all the time) is "wrong." It's really a way to define the effect of rules differences, or training differences, or whatnot. For example, a league total change is required to make the league behave properly to the late 60s process of lowering the mound and reducing the strike zone area. A league total change is required to reflect the rule change about marring the ball, or to reflect lighter/heavier bats. Play a league on Mars, and a league total change is required. This is a real issue from a modeling perspective. League totals are attempting to do two things: (1) Define the quality of players in the league, and (2) define the special environmental rules in effect. You can tell this because you have to go to league totals in order to influence either historical baseball era settings, or to influence minor league performance in any league. This is a flaw, actually. I'm not educated enough on the actual algorithms to know what to do to correct it, but it seems intuitive that Markus could and should get "league average" by doing a true average of the players in the league at any time--this would be calculated in the background and perhaps displayed in a set-up screen. This would remove that aspect of the model fromthe league totals, leaving only an "Era/Environmental adjustment" setting. I think this would make the whole thing less confusing. 4) What have we done to the model to create these stats? Since the OOTP League Totals model includes "expected quality of the players in the league" it is assuming some things about the ratings of players in the league. In other words, if the average quality (ratinngs) of the players in the league are in the ballpark of the league totals, you'll get those results. What we've done, though, is tell the game engine that we've got a league of a level of quality that is far different from the actual players in the league. This means that almost every AB is being made by a player that the game engine views as being an outlier. If the game engine thinks every hitter is effectively made into a Steve Jeltz, or a Babe Ruth, then you can see that things will get out of whack quickly. So essentially we are either lying to the game engine with league totals by telling it that the average player is far out of whack with reality, or by entering these skewed League Totals we're telling the game engine tha the rules/environment we've selected to play in is very different from "real baseball." All that said, there are still things to learn from this set of league data... 5) When you adjust the AB Totals, you can see how the game has implemented the DIPS theory. DIPS says that pitchers control only strikeouts, walks, and HR. So it's interesting to look at these numbers as we adjust AB. When we have High AB League Totals, we find that strikeouts soar. Remember that every result is influenced by pitcher and hitter. So this is because the "league average pitcher" is very poor, so a major league pitcher will strike out a ton of hitters in this environment. Similarly, walks soar because the league average hitter will almost never walk, and the average major league hitter and pitcher walk and give up more walks. Home runs also soar, for the same reason. Of more interest, though, is BABIP. BABIP in this environment is an astronomical .898/.890. So almost every result is either a hit or a strikeout. Similarly, when we lower the AB League total we find the counter to be true--BABIP drops to a paltry .149/.152. What the heck? We didn't change the BABIP setting at all. How can this be? 6) What's going on with BABIP? Time for a caveat. I really know nothing about how the game engine does its thing. Markus has described it before, but in the end only he knows the truth of the engine. But here's my theory, the results engine is not as robust as it needs to be because it determines results in the wrong order. I've said this lots of times, and shown thought-logic to defend my position. But the data above, in my mind, confirms this beyond any shadow of a doubt. Two other things could be aggravating this -- (1) the that BABIP rating could be mapped to values that are too high, but I would guess that's not the case as Markus is ususally pretty diligent about those kinds of things and that would be a moderately big and obvious miss. (2) Markus could still be using the Contact rating to determine events, but I don't think he is. Let's look at some additional data: Code: League Batting Stats AB/G H/AB 2B/AB 3B/AB HR/AB BB/AB SO/AB BABIP 2B/H 3B/H HR/H 2008 42.4 .411 .089 .011 .071 .744 .551 .898 .216 .026 .173 2009 41.9 .404 .085 .010 .068 .745 .554 .889 .211 .025 .168 2010 33.3 .153 .038 .004 .020 .054 .089 .149 .249 .025 .133 2011 33.0 .155 .039 .004 .020 .052 .086 .152 .248 .023 .128 League Pitching Stats H/9 HR/9 BB/9 K/9 K/BB 2008 17.9 3.1 32.5 24.0 .740 2009 17.4 2.9 32.1 23.9 .744 2010 4.8 0.6 1.7 2.8 1.663 2011 4.9 0.6 1.6 2.7 1.637 In 2008/2009 you see that everything happens more often--pitchers and hitters walk more and strike out lots more (24 Ks/game mean only three outs happen on Balls in Play). Teams hit 3 HR/game. We constrain K, HR, and BB (which I think get calculated first), so all the game engine has to work with are balls in play. You can see that the game engine is doing its best in the 2008/2009 ears to make hitters really good. So either the game is trying to control batting average or BABIP. Since BABIP is fluctuating oddly, I assume it's attempting to control average--which makes sense given that the game is determining if a result is a hit too early in the process. Note, also, the three columns to the right (2B/H, 3B/H, and HR/H). Note espeically that our 2B/H and HR/H ratios behave in inverse when the totals flip-flop in dramatic fashion. By this I mean the numbers of 2B/H increase while the numbers of HR/AB decrease, and visa versa. I think this is an important indicator of something, and could speculate on it ... but only Markus would know for sure. 7) Last thoughts This exercise is so interesting because it's stretching the game engine a couple ways--it's saying hitters are better or worse in the same breath (increasing BB, SO, HR all at the same time), and the same for pitchers (increasing BB, SO, and HR). It gives you lots to think about, and shows exactly how complex the game engine really is. It also exposes a few places where it struggles to provide results due to the model's abstraction being made at too high of a level to properly deal with more detailed issues. This is the case for pretty much all baseball simulations (think of the major abstractions in our old dice or spinner games), but it drives a lot of confusion when you get into the details. I recommend that sometime in the future Markus: 1) Calculate the real league average player rating and use that in the definition of League Totals (I'm sure this will be complex, and cause further issues with the historical/fictional crowds) 2) Provide a "Era Adjustments" screen rather than league totals 3) Add the use of GB/FB/LD data to the early stages of the results engine, and determine defense, hit, and HR events at the proper locations (again, I'm sure this is complex). Last edited by RonCo; 07-13-2008 at 11:45 AM.