OOTP Developments Forums - View Single Post

Garlon · 08-27-2007, 04:31 PM

Hello fellow historical simmers,

Spritze and I started working on a new modified historical DB for OOTP recently. Spritze has done 99% of the work on it and has made (and is still in the process of making) many meticulous edits to provide the best results with OOTP.

The biggest modification we have made with the DB is that it will have park-neutralized player stats. We are using the method described by Bill James in his Historical Abstract to both park-neutralize player stats and convert them to a 750-run environment. If you go to baseball-reference.com you can look up any player and click an option to neutralize stats to a 750-run park-neutral environment. The stats in our DB will essentially be the same as what you would find there since we are using the same conversion technique.

Don't be too concerned with the fact that teams weren't scoring 750-runs per season in the Deadball Era and that this conversion will have a negative impact on simulation results. In fact, just the opposite will happen and we willl see even better results across all eras. OOTP's League Totals Modifiers control the environment and ratios of things like BA and HR per AB, the conversion of the stats in our DB to a higher run-scoring environment just changes player ratings...not the results of the simulation. The important aspect of player ratings in OOTP is how players compare relative to one another, not the absolute value of the ratings. This park-neutral 750-run conversion puts all players on an equal footing for comparison so that players who played for the Dodgers in Dodger Stadium for instance who have poor hitting stats because that is perhaps the worst hitter's park in baseball will benefit from the conversion, while players who played in Coors Field have their statistics reduced. Willie Davis who played fo the Dodgers 1960-1979 had a career .279 BA, after conversion he goes to .301. This is the primary advantage this DB will provide - better representation of a player's abilities.

We are also normalizing player fielding stats to their year and league. Players who played on teams with high strikeout pitching staffs had fewer opportunities to make plays in the field. We adjust player Put Out and Assist totals to adjust for things like this. We also adjusted Double Play totals of all players on teams based on thei teams Expected Double Plays Per Game compared to the League Average Double Plays Per Game of that season. Both of these adjustment formulas come from Bill James' book Win Shares.

Readme/FAQ

1. Fixed missing batter Strikeout data.
American Association 1882-1888 and 1890
National League 1897-1909
American League 1901-1912

2. Fixed missing Stolen Base data 1876-1885.
Stolen Bases were set to the Deadball Era frequency on a per times reaching first base basis.

3. Adjusted Stolen Base data 1871-1875 and 1886-1897.
Due to differences in the definition of a stolen base before 1898, stolen bases during these periods were corrected to the frequency of the Deadball Era.

4. Fixed missing Caught Stealing 1871-1919 NL/AL and 1926-1950 NL.
For the 1871-1919 period, league SB% was set to a 55% success rate.
A formula was used so that players would range above and below 55% based on their frequency of scoring runs.
In general, better players have higher success rates, but the league as a whole remains at 55%.

For the 1926-1950 period in the NL, league SB% was set to match the SB% of the AL in any given season.
The same formula was used to give higher success rates to better players.

5. Deleted any playes who debuted in the following leagues and did not have at least 550AB or 162IP in the real major leagues.
Union Association 1884
Players League 1890
Federal League 1914-1915

This was done to prevent OOTP leagues from being flooded with non-major league talent.

6. Deleted any WWII replacement players who debuted between 1942-1945 who did not have at least 550 AB or 162 IP after the 1945 season.
This was done to prevent OOTP leagues from being flooded with non-major league talent.

7. Pro-rated any missed seasons during WWII for the 400+ players who served in the military.
Their estimated stats are based on their performance before and after WWII.

8. Fixed gaps in the playing records of all players in the DB.
Many players missed seasons for various reasons during their career, being sent to the minor leagues, injuries, military service, etc.
We filled in these gaps of a player's recod by taking an average of the bookend seasons of the gap and applying it to all missing seasons in that gap.
This was done so that no players would be permanently missing when you start a league in OOTP.
It will also help with recalc mode by providing additional statistics.

9. Fixed low AB or IP totals for players who had at least one full season of career stats but may have only had a few AB or IP in a given season.
We pro-rated any missing AB to give a player 251 AB on the season based on his career average.
We pro-rated any missing IP to give a pitcher 40 IP on the season based on his career average.
This was done so that OOTP can provide better ratings in these instances.
All players in the DB who had at least 550 AB career or 162 IP career, will have at least 251 AB and 40 IP in any season in which they played.

10. Adjusted cup-of-coffee players to replacement level based on the seasons and their position.
This was done becaue these players do no have a large enough sample of AB or IP for OOTP to create valid ratings. We would rather prescribe the replacement level stats than have OOTP do it.

11. Limited the # of Starting Pitchers in the DB.
Set the Games Started to 0 for all pitchers with fewer than 125 career Games Started.
This was done to keep pitchers who had very short careers from stealing rotation spots from more deserving ptichers in OOTP.
These pitchers will now import as Relievers with an Endurance of roughly 2 IP.

12. Added extra player ID's for players who had significant careers as both a position player and a hitter, like Babe Ruth.

13. Extended the careers of significant post-integration players.
Due to segregation in baseball until 1947, many players like Jackie Robinson did not have a chance to play in the major leagues at a younger age.
These players had their careers extended backwards so that they now debut at age 21.
Campanella, Doby, Easter, Irvin, Minoso, Robinson.

14. Added 50 Negro League stars to the DB.
These players were given the career average stats based on MLE's of their available Negro league stats when possible. Otherwise we base their stats on comparable major league HOF'ers.

The Negro League Players by position, their career span, and their major league equivalents:

c - Clarence Wlliams (1889-1905) - Ewing
c - Louis Santop (1911-1927) - Bresnahan
c - Biz Mackey (1919-1935) - Cochrane
c - Ted Radcliffe (1924-1940) - R. Ferrell
c - Josh Gibson (1933-1949) - Foxx

1b - John Frye (1883-1899) - Connor
1b - Ben Taylor (1910-1926) - G. Kelly
1b - Buck Leonard (1929-1945) - Mize
1b - Buck O'Neil (1933-1949) - Terry

2b - Frank Grant (1887-1903) - McPhee
2b - Bingo DeMoss (1911-1927) - Evers
2b - Newt Allen (1923-1939) - Frisch
2b - Sammy Hughes (1932-1948) - Bi. Herman

3b - Sol White (1890-1906) - J. Collins
3b - Candy Jim Taylor (1905-1921) - Baker
3b - Jud Wilson (1915-1931) - Lindstrom
3b - Judy Johnson (1921-1937) - Traynor
3b - Ray Dandridge (1935-1951) - Kell

ss - Grant Johnson (1896-1912) - G. Davis
ss - Pop Lloyd (1906-1922) - Wagner
ss - Dick Lundy (1920-1936) - T. Jackson
ss - Willie Wells (1927-1943) - Appling

lf - George Williams (1885-1901) - McCarthy
lf - Pete Hill (1902-1918) - Clarke
lf - Mule Suttles (1922-1938) - Wilson
lf - Turkey Stearnes (1923-1939) - Klein

cf - Bud Fowler (1879-1905) - O’Rourke
cf - Spotswood Poles (1911-1927) - Carey
cf - Oscar Charleston (1918-1934) - Heilmann
cf - Cool Papa Bell (1925-1941) - Cuyler

rf - Benjamin Boyd (1880-1896) - K. Kelly
rf - Cristobal Torriente (1916-1938) - S. Rice
rf - Martin Dihigo (1927-1943) - L. Waner
rf - Willard Brown (1935-1951) - Slaughter

p - William Whyte (1882-1898) - Clarkson
p - George Parago (1883-1899) - Radbourn
p - William Malone (1887-1903) - Rusie
p - George Stovey (1887-1903) - Young
p - Rube Foster (1901-1917) - Plank
p - Smokey Joe Wlliams (1907-1923) - Waddell
p - Jose Mendez (1908-1924) - Marquard
p - Bullet Joe Rogan (1911-1927) - Rixey
p - Andy Cooper (1918-1934) - Grimes
p - William Bell (1919-1935) - Haines
p - Bill Foster (1926-1942) - Lyons
p - Satchel Paige (1928-1944) – Grove
p - Chet Brewer (1928-1944) - C. Hubbell
p - Ray Brown (1929-1945) - Gomez
p - Hilton Smith (1933-1949) - Di. Dean
p - Leon Day (1938-1954) - Feller

15. Fixed miscellaneous problems in the DB files.
Erased the pitching stats of non-pitchers.
Erased the non-pitcher fielding stats of pitchers.
Created career average hitting stats for pitchers.
Fixed missing birthdays, height, weight, bats, throws fields in the master file.

16. Converted Batting and Pitching statistics of all players to a 750-run environment.
This conversion will give the best representation of a player's skills in a park-neutralized environment converted to a league average scoring of 4.63 runs per team game.
This does not mean that players who have their statistics boosted will become sperior players than what they should be.
OOTP's league totals modifier system keeps league totals in check, so you will not be scoring 4.63 runs per game in the Deadball Era even though we adjusted the stats this way.
The translation of the stats gives a better representation of a player's abilities relative to the other players in the league by eliminating park effects.

17. Adjusted the fielding statistics of all players.
We normalized the Put Outs and Assist totals of alll players based on the balls in play per game of their team versus the balls in play per game average of their league.
This was done so that players on teams whose pitchers did not strike out many batters do not get superior defensive ratings in OOTP just because they had more opportunities to make plays.
We adjusted Errors to the new Put Out+ Assist totals so that a player's FLD% remained the same after the conversion.
We normalized Double Play statistics based on a teams Expected Double Plays per Game versus the league's Double Plays per Game.
This was done so that players on teams which allowed alot of base runners from Hits and BB do not receive superior Turn DP ratings in OOTP due to their extra opportunities.

18. Added 14 Japanese greats to the DB.
Except for Sadaharu Oh, we have not selected these yet.

19. Extended backwards the careers of modern Japanese players in MLB.
This way players like Ichiro Suzuki and Hideo Nomo will start their careers at age 21.

08-27-2007, 04:31 PM	#1 (permalink)
Garlon All Star Starter Join Date: Jun 2004 Posts: 1,551 Thanks: 0 Thanked 52x in 26 posts	Spritze/Garlon DB Hello fellow historical simmers, Spritze and I started working on a new modified historical DB for OOTP recently. Spritze has done 99% of the work on it and has made (and is still in the process of making) many meticulous edits to provide the best results with OOTP. The biggest modification we have made with the DB is that it will have park-neutralized player stats. We are using the method described by Bill James in his Historical Abstract to both park-neutralize player stats and convert them to a 750-run environment. If you go to baseball-reference.com you can look up any player and click an option to neutralize stats to a 750-run park-neutral environment. The stats in our DB will essentially be the same as what you would find there since we are using the same conversion technique. Don't be too concerned with the fact that teams weren't scoring 750-runs per season in the Deadball Era and that this conversion will have a negative impact on simulation results. In fact, just the opposite will happen and we willl see even better results across all eras. OOTP's League Totals Modifiers control the environment and ratios of things like BA and HR per AB, the conversion of the stats in our DB to a higher run-scoring environment just changes player ratings...not the results of the simulation. The important aspect of player ratings in OOTP is how players compare relative to one another, not the absolute value of the ratings. This park-neutral 750-run conversion puts all players on an equal footing for comparison so that players who played for the Dodgers in Dodger Stadium for instance who have poor hitting stats because that is perhaps the worst hitter's park in baseball will benefit from the conversion, while players who played in Coors Field have their statistics reduced. Willie Davis who played fo the Dodgers 1960-1979 had a career .279 BA, after conversion he goes to .301. This is the primary advantage this DB will provide - better representation of a player's abilities. We are also normalizing player fielding stats to their year and league. Players who played on teams with high strikeout pitching staffs had fewer opportunities to make plays in the field. We adjust player Put Out and Assist totals to adjust for things like this. We also adjusted Double Play totals of all players on teams based on thei teams Expected Double Plays Per Game compared to the League Average Double Plays Per Game of that season. Both of these adjustment formulas come from Bill James' book Win Shares. Readme/FAQ 1. Fixed missing batter Strikeout data. American Association 1882-1888 and 1890 National League 1897-1909 American League 1901-1912 2. Fixed missing Stolen Base data 1876-1885. Stolen Bases were set to the Deadball Era frequency on a per times reaching first base basis. 3. Adjusted Stolen Base data 1871-1875 and 1886-1897. Due to differences in the definition of a stolen base before 1898, stolen bases during these periods were corrected to the frequency of the Deadball Era. 4. Fixed missing Caught Stealing 1871-1919 NL/AL and 1926-1950 NL. For the 1871-1919 period, league SB% was set to a 55% success rate. A formula was used so that players would range above and below 55% based on their frequency of scoring runs. In general, better players have higher success rates, but the league as a whole remains at 55%. For the 1926-1950 period in the NL, league SB% was set to match the SB% of the AL in any given season. The same formula was used to give higher success rates to better players. 5. Deleted any playes who debuted in the following leagues and did not have at least 550AB or 162IP in the real major leagues. Union Association 1884 Players League 1890 Federal League 1914-1915 This was done to prevent OOTP leagues from being flooded with non-major league talent. 6. Deleted any WWII replacement players who debuted between 1942-1945 who did not have at least 550 AB or 162 IP after the 1945 season. This was done to prevent OOTP leagues from being flooded with non-major league talent. 7. Pro-rated any missed seasons during WWII for the 400+ players who served in the military. Their estimated stats are based on their performance before and after WWII. 8. Fixed gaps in the playing records of all players in the DB. Many players missed seasons for various reasons during their career, being sent to the minor leagues, injuries, military service, etc. We filled in these gaps of a player's recod by taking an average of the bookend seasons of the gap and applying it to all missing seasons in that gap. This was done so that no players would be permanently missing when you start a league in OOTP. It will also help with recalc mode by providing additional statistics. 9. Fixed low AB or IP totals for players who had at least one full season of career stats but may have only had a few AB or IP in a given season. We pro-rated any missing AB to give a player 251 AB on the season based on his career average. We pro-rated any missing IP to give a pitcher 40 IP on the season based on his career average. This was done so that OOTP can provide better ratings in these instances. All players in the DB who had at least 550 AB career or 162 IP career, will have at least 251 AB and 40 IP in any season in which they played. 10. Adjusted cup-of-coffee players to replacement level based on the seasons and their position. This was done becaue these players do no have a large enough sample of AB or IP for OOTP to create valid ratings. We would rather prescribe the replacement level stats than have OOTP do it. 11. Limited the # of Starting Pitchers in the DB. Set the Games Started to 0 for all pitchers with fewer than 125 career Games Started. This was done to keep pitchers who had very short careers from stealing rotation spots from more deserving ptichers in OOTP. These pitchers will now import as Relievers with an Endurance of roughly 2 IP. 12. Added extra player ID's for players who had significant careers as both a position player and a hitter, like Babe Ruth. 13. Extended the careers of significant post-integration players. Due to segregation in baseball until 1947, many players like Jackie Robinson did not have a chance to play in the major leagues at a younger age. These players had their careers extended backwards so that they now debut at age 21. Campanella, Doby, Easter, Irvin, Minoso, Robinson. 14. Added 50 Negro League stars to the DB. These players were given the career average stats based on MLE's of their available Negro league stats when possible. Otherwise we base their stats on comparable major league HOF'ers. The Negro League Players by position, their career span, and their major league equivalents: c - Clarence Wlliams (1889-1905) - Ewing c - Louis Santop (1911-1927) - Bresnahan c - Biz Mackey (1919-1935) - Cochrane c - Ted Radcliffe (1924-1940) - R. Ferrell c - Josh Gibson (1933-1949) - Foxx 1b - John Frye (1883-1899) - Connor 1b - Ben Taylor (1910-1926) - G. Kelly 1b - Buck Leonard (1929-1945) - Mize 1b - Buck O'Neil (1933-1949) - Terry 2b - Frank Grant (1887-1903) - McPhee 2b - Bingo DeMoss (1911-1927) - Evers 2b - Newt Allen (1923-1939) - Frisch 2b - Sammy Hughes (1932-1948) - Bi. Herman 3b - Sol White (1890-1906) - J. Collins 3b - Candy Jim Taylor (1905-1921) - Baker 3b - Jud Wilson (1915-1931) - Lindstrom 3b - Judy Johnson (1921-1937) - Traynor 3b - Ray Dandridge (1935-1951) - Kell ss - Grant Johnson (1896-1912) - G. Davis ss - Pop Lloyd (1906-1922) - Wagner ss - Dick Lundy (1920-1936) - T. Jackson ss - Willie Wells (1927-1943) - Appling lf - George Williams (1885-1901) - McCarthy lf - Pete Hill (1902-1918) - Clarke lf - Mule Suttles (1922-1938) - Wilson lf - Turkey Stearnes (1923-1939) - Klein cf - Bud Fowler (1879-1905) - O’Rourke cf - Spotswood Poles (1911-1927) - Carey cf - Oscar Charleston (1918-1934) - Heilmann cf - Cool Papa Bell (1925-1941) - Cuyler rf - Benjamin Boyd (1880-1896) - K. Kelly rf - Cristobal Torriente (1916-1938) - S. Rice rf - Martin Dihigo (1927-1943) - L. Waner rf - Willard Brown (1935-1951) - Slaughter p - William Whyte (1882-1898) - Clarkson p - George Parago (1883-1899) - Radbourn p - William Malone (1887-1903) - Rusie p - George Stovey (1887-1903) - Young p - Rube Foster (1901-1917) - Plank p - Smokey Joe Wlliams (1907-1923) - Waddell p - Jose Mendez (1908-1924) - Marquard p - Bullet Joe Rogan (1911-1927) - Rixey p - Andy Cooper (1918-1934) - Grimes p - William Bell (1919-1935) - Haines p - Bill Foster (1926-1942) - Lyons p - Satchel Paige (1928-1944) – Grove p - Chet Brewer (1928-1944) - C. Hubbell p - Ray Brown (1929-1945) - Gomez p - Hilton Smith (1933-1949) - Di. Dean p - Leon Day (1938-1954) - Feller 15. Fixed miscellaneous problems in the DB files. Erased the pitching stats of non-pitchers. Erased the non-pitcher fielding stats of pitchers. Created career average hitting stats for pitchers. Fixed missing birthdays, height, weight, bats, throws fields in the master file. 16. Converted Batting and Pitching statistics of all players to a 750-run environment. This conversion will give the best representation of a player's skills in a park-neutralized environment converted to a league average scoring of 4.63 runs per team game. This does not mean that players who have their statistics boosted will become sperior players than what they should be. OOTP's league totals modifier system keeps league totals in check, so you will not be scoring 4.63 runs per game in the Deadball Era even though we adjusted the stats this way. The translation of the stats gives a better representation of a player's abilities relative to the other players in the league by eliminating park effects. 17. Adjusted the fielding statistics of all players. We normalized the Put Outs and Assist totals of alll players based on the balls in play per game of their team versus the balls in play per game average of their league. This was done so that players on teams whose pitchers did not strike out many batters do not get superior defensive ratings in OOTP just because they had more opportunities to make plays. We adjusted Errors to the new Put Out+ Assist totals so that a player's FLD% remained the same after the conversion. We normalized Double Play statistics based on a teams Expected Double Plays per Game versus the league's Double Plays per Game. This was done so that players on teams which allowed alot of base runners from Hits and BB do not receive superior Turn DP ratings in OOTP due to their extra opportunities. 18. Added 14 Japanese greats to the DB. Except for Sadaharu Oh, we have not selected these yet. 19. Extended backwards the careers of modern Japanese players in MLB. This way players like Ichiro Suzuki and Hideo Nomo will start their careers at age 21.