Arod23/Garlon Database Developer Blog

[Garlon]

Arod23Garlon2006DB now available. Download it here

Hello OOTP simmers,

Arod23 and I are currently starting work on a new database for OOTPBM2006. We are starting completely from scratch and are planning a variety of edits and improvements over our previous database.

I will update this thread with our progress as we make changes step-by-step. Any feedback/suggestions will also be welcome.

So I'll start with the fist edits I made today:

Strikeout data for batters were missing in the American Association from 1882-1888 and 1890. From National League from 1897-1909. And from the American League from 1901-1912. However, we do have league strikeout totals for all of these years by league. So what I did was determine the ratio of outs that were strikeouts for each league each year and then apply this ratio to the number of outs that each batter made. Outs are determined by At Bats minus Hits. I'll give you an example:

In the AL in 1907 there were 4479 strikeouts and 30833 outs made, so 14.52% of all outs made were strikeouts. Ty Cobb had 212 hits in 605 ABs, therefore making 393 outs. So we estimate his strikeouts at 14.52% of 393 (393 x .1452) which gives us 57 strikeouts for Ty Cobb in 1907.

Thanks

[tim_uwrf]

Is this a Lahman database like database?

[Garlon]

Yes it is Tim.

[Garlon]

I just finished more edits. These affect stolen base and caught stealing data.

Stolen Base data is not available from 1876-1885. So I estimated the frequency of successful stolen bases during this 10-year period by looking at data from 1886-1895, which is the very next 10 years. I calculated the frequency of stolen bases per times on first base for 1886-1895...to do this I found the ratio of stolen bases per times reaching first base. Times reaching first base was calculated as Hits minus doubles minus triples minus homeruns plus bases on balls [(H-2b-3b-HR)+BB]. I know that players were hitting doubles and then stealing 3rd base, or reaching first base on fielder's choice, but I do not have any practical way of estimating such a frequency of these things occuring...so my method just assumes that stolen bases occurred when a player reached 1b by a single or a base on balls. From 1886-1895, players successfully stole one base per 5 times they reached 1b. So for 1876-1885 I multiplied all players times on 1b by .20 (20%) to estimate their stolen bases.

Caught Stealing data is missing for the NL from 1876-1914, 1916-1919, and 1926-1950. For the AA from 1882-1890. And from the AL from 1901-1913, and 1916-1919.

Missing Caught Stealing data for all years before 1926 was based on actual existing data from the AL 1914-1915, the NL 1915, and the NL and AL from 1920-1925. What I found was that players were stealing successfully about 55% of the time during this time frame, so I applied this 55% success rate to all missing data before 1926.

For the missing NL caught stealing data from 1926-1950, I based each year on the AL's caught stealing % year by year. So in 1926 when the AL was 57% successful at stealing bases, so was the NL now 1926...in 1949 when the AL was 54.8% successful, so was the NL now in 1949.

[Garlon]

Edits regarding the Union Association, Players League, and Federal League:

The Union Association existed for only a single season, 1884. This was not much of a "real" major league since only 14 of its players actually had a major league career outside of this league. We will only be keeping these 14 players and deleting the rest. On top of that, I have adjusted the batting stats of the UA players to the ML averages of 1884. For example, in 1884 the real Major Leagues hit .2426 and the UA hit .2449....2426/.2449 = .9906, therefore, I multiplied all hits in the UA by .9906 so that the UA league batting average is now .2426.

This same process was done for the Players League of 1890 and the Federal League of 1914-1915. Although we recognize the PL and FL as real major leagues, the level of offense was substantially higher than in the other Major Leagues of the time. This is why we adjust their statistics to bring them in line with the other Major Leagues. For example, in 1914 the other major leagues were only hitting HR's 70.42% as often per at bat as the FL, so all FL 1914 HR totals have been multiplied by .7042, so that they now hit HRs at the same rate as the rest of the Major Leagues.

[Garlon]

Slight correction to my first post - strikeout data was missing from the American Association from 1882-1890.

We are planning to include Negro Leaguers in our database. We have not decided yet if we will have the top two negro leaguers per position and 10 pitchers (26 players) or if we will go to three negro leaguers per position and 12 pitchers (36 players) and try to have a negro league player for each position during each decade from the deadball era up until the integration of baseball in the 1940's. An example of this idea for catcher would be:

Louis Santop (1911-1925)
Biz Mackey (1919-1933)
Josh Gibson (1933-1947)

As you can see with these three players there will always be some Negro League catcher present in the baseball from 1911 thru the 1940's. Or we could just go with two of these guys.

I will put up a post soon with players for each position that are being considered as well as how we plan to handle the Major League Equivalncies for the Negro Leaguers.

[Garlon]

World War II 1942-1945 War Years edits:

These years are modified because WWII drew many major league players into military service thus thinning out the talent in the leagues, plus the baseballs were manufactured poorly during this period. I have just adjusted all of the batting stats from 1942-1945 so that these years now play at the level that major league baseball did from 1938-1941 + 1946-1949. For instance, in 1942 HRs were down 37% compared to '38-'41 and '46-'49, so I multiplied everyone's HRs by 1.37. Along with this edit we will also be inserting any missed seasons for players who were in the military...for instance, Joe DiMaggio missed 1943-1945 serving in the military - these missing seasons will be calculated by inserting his average for every offensive category based on what he did between 1938-1949. This combination of edits should allow for more realistic simulating through the 1940's.

[Garlon]

Negro League Players:

One of the last things we plan to do is introduce a number of Negro League players into our database. We did this in our last database, but we are taking a fresh approach to it this time and we are planning to rework the Major League Equivalencies for many of the players. The first thing we need to do is decide how many players to put in and who they are. In or last database we used 26 players - 2 players per position, plus 10 pitchers. This time I am considering broadening the scope to 3 players per position plus 12 pitchers, for a total of 36 Negro League players. For our original list of 2 players per position we basically used the top 2 players at each position as Bill James has them ranked in his Historical Abstract. For this next database I thought it might be a good idea to go to 3 players per position, but also have a Negro League player at each position per era from the Deadball Era until integration.

Here is a list of the players I have tentatively chosen by position. Each player will debut at age 21 and be given a 15 year career...the dates after the player's name is his theoretical career.

Catcher
Louis Santop (1911-1925)
Biz Mackey (1919-1933)
Josh Gibson (1933-1947)

Firstbase
Ben Taylor (1910-1924)
Buck Leonard (1929-1943)
Buck O'Neil (1933-1947)

Secondbase
Bingo DeMoss (1911-1925)
Newt Allen (1923-1937)
Sammy Hughes (1932-1946)

Thirdbase
Candy Jim Taylor (1905-1919)
Judy Johnson (1921-1935)
Ray Dandridge (1935-1939)

Shortstop
Pop Lloyd (1906-1920)
Dick Lundy (1920-1934)
Willie Wells (1927-1941)

Leftfield
Pete Hill (1901-1915)
Mule Suttles (1922-1936)
Turkey Stearnes (1923-1937)

Centerfield
Cristobal Torriente (1916-1930)
Oscar Charleston (1918-1932)
Cool Papa Bell (1925-1939)

Rightfield
Spotswood Poles (1908-1922)
Martin Dihigo (1927-1941)
Willard Brown (1935-1949)

Pitchers
Rube Foster (1901-1915)
Smokey Joe Williams (1907-1921)
Jose Mendez (1908-1922)
Bullet Joe Rogan (1911-1925)
Andy Cooper (1918-1932)
William Bell (1919-1933)
Bill Foster (1926-1940)
Satchel Paige (1928-1942)
Chet Brewer (1928-1942)
Ray Brown (1929-1943)
Hilton Smith (1933-1947)
Leon Day (1938-1952)

What about having one Negro League player per position who played pre-1900? I don't know too much about pre-1900 Negro League Baseball...I think the Cuban Giants which started in 1885 was the best team, and perhaps the only salaried/professional team. Here are some pre-1900 Negro League player ideas:

c - Moses Fleetwood Walker (already in the database, but could modify his totals to a full career since he only played 42 games)
1b - John Frye
2b - Frank Grant
3b - Sol White
ss - Grant Homerun Johnson
of - George Williams
of - Benjamin boyd
of - Harry Johnson
p - Bud Fowler
p - George Stovey

[Garlon]

Ted Williams and the Korean War:

I just adjusted Ted Williams 1952-1953 batting totals to his average totals from '50-'51 + '54-'55 while also incorporating the few at bats he actualy did have in '52-'53. This was done because Ted Williams missed nearly all of those seasons due to military service in the Korean War.

If anyone has a reference or a listing of players who missed all or parts of seasons due to service in the Korean War or World War I please let me know so that I can modify player stats if necessary.

Thanks

[Garlon]

I just finished adjusting the pitching statistics of the Union association 1884, the Players League 1890, the Federal League 1914-1915, and the War Years 1942-1945 to the appropriate major league levels.

[redsox45]

WOW!!

This sounds like it will be an incredible database when it is done.

What (if any) are your plans for adding historical minor leaguers? I know that would be asking a lot, but just wondering if you have any plans to do that sort of thing?

Will

[catcher24]

Each player will debut at age 21 and be given a 15 year career...

Might I be so bold as to suggest that you debut them at 19 or 20 (not unusual, especially for Negro League players) and give them an 18 to 20 year career? Many of them played year round (Negro Legue, Mexican League, Winter Leagues), and many of them played well into their 40's. A lot of Negro Leaguers played 200+ games per year. I don't think it would be pushing the bounds of reality to give them a little longer career. Please take this as a suggestion, not a criticism. You have undertaken a huge project which will be of tremendous benefit to all users of OOTP, and I thank you for it!

[Garlon]

redsox45 - Sorry there won't be any historical minor leaguers. It's just a major league database.

catcher24 - We are probably going to stick with having them debut at age 21. Many of the Negro League players did as you say start playing even before they were 21 years old. However, it is hard to believe they would have been ready for the major leagues at that age. Some of them may not have even started until they were older than 21. Also these players will probably be based on comparable major league players, probably all of which will be Hall Of Famers - so these Negro League players will be very strong as it is without giving them an extra adavantage of having them start at 18 or 19 years old. I think 21 years old is a good compromise. As for giving them a 15-year career, well it seems that many players start to seriously decline in OOTP after age 35 anyway, although occasionally I see players still playing at all-star level at 40 years old. So 15 years I think is a reasonable span for estimating how long a quality player plays every day. Also, regardless of how many years I give them in the database they may play longer in your own simulation.

[catcher24]

Sounds reasonable. Look forward to it.

[Garlon]

Possible MLE's (Major League Equivalencies) for Negro League Players.

When we introduce the Negro League Players into the Database we will base their statistics on comparable major league players.
These comparable players must meet 3 criteria:
1. They played during the same time period as the Negro Leaguer
2. They played the same position as the Negro Leaguer
3. They are members of the Hall Of Fame

The Negro League Players stats for each season will be the career average of their MLE, except for their rookie season which will be reduced by some %, perhaps 75% of the career average.

Here is our tentative list:

Catcher
Louis Santop (1911-1925) = Bresnahan or Schalk
Biz Mackey (1919-1933) = Cochrane or Hartnett
Josh Gibson (1933-1947) = Foxx

Firstbase
Ben Taylor (1910-1924) = Chance or Kelly or Sisler
Buck Leonard (1929-1943) = Mize
Buck O'Neil (1933-1947) = Terry

Secondbase
Bingo DeMoss (1911-1925) = Evers
Newt Allen (1923-1937) = Frisch
Sammy Hughes (1932-1946) = Herman or Doerr or Lazzeri

Thirdbase
Candy Jim Taylor (1905-1919) = Baker or J. Collins
Judy Johnson (1921-1935) = Traynor or Lindstrom
Ray Dandridge (1935-1939) = Kell

Shortstop
Pop Lloyd (1906-1920) = Wagner
Dick Lundy (1920-1934) = T. Jackson or Sewell
Willie Wells (1927-1941) = Cronin or Appling or Vaughan

Leftfield
Pete Hill (1901-1915) = Clarke or Kelley
Mule Suttles (1922-1936) = Wilson
Turkey Stearnes (1923-1937) = Klein

Centerfield
Cristobal Torriente (1916-1930) = Roush or Rice
Oscar Charleston (1918-1932) = Heilmann
Cool Papa Bell (1925-1939) = Cuyler

Rightfield
Spotswood Poles (1908-1922) = Carey or Hooper
Martin Dihigo (1927-1941) = L. Waner or P. Waner or Hafey
Willard Brown (1935-1949) = Medwick or Slaughter

Pitchers
Rube Foster (1901-1915) = Plank
Smokey Joe Williams (1907-1921) = Waddell
Jose Mendez (1908-1922) = Marquard
Bullet Joe Rogan (1911-1925) = Rixey
Andy Cooper (1918-1932) = Grimes
William Bell (1919-1933) = Hoyt or Vance
Bill Foster (1926-1940) = Hubbell
Satchel Paige (1928-1942) = custom
Chet Brewer (1928-1942) = Lyons
Ray Brown (1929-1943) = Gomez
Hilton Smith (1933-1947) = Dean
Leon Day (1938-1952) = Feller or Wynn

[catcher24]

Looks interesting. I would probably equate Oscar Charleston to someone with more power than Heilmann, but not sure who. He was compared to Speaker as a fielder, Cobb as a baserunner, and Ruth for power, but how do you combine all those facets? As for Paige, perhaps a good match (rather than custom) would be Lefty Grove.

[Garlon]

catcher24 - Oscar Charleston reputedly was a great power hitter. It can be difficult to find a perfect match for some of these guys. But Heilmann's BA/OBP/SLG for his career were .342/.410/.520 So he had a .930 OPS. With Suttles and Stearnes debuting in the early 1920's I didn't want to completely flood the league with HRs. In our original database, Charleston was based on Tris Speaker. Speaker's career averages are .345/.428/.500, very close to Heilmann's. I picked Heilmann's this time because his career's time frame more closely resembles Charleston's than Speaker's does.

Update: I am beginning the process of adjusting all years in the database to a 162-game environment. This primarily impacts pre-1901 baseball and will theoretically result in more accurate talent ratings for players who played in times of a fluctuating schedule...without this adjustment a player may have had his best years during seasons in which the schedule ranged from 70-100, then in his later years when he was declining the schedule also increased to 132 games...thereby giving him more playing time and ABs in his less productive years which will weight his ratings more toward this end of his career which may not be a true representations of a player's abilities since he did not have the opportunity to rack up as many ABs during his prime.

I just finished adjusting the National Association 1871-1875 batting stats to a 162-game environment.

[Garlon]

I just finished adjusting batting stats for all seasons 1876-present to a 162-game environment.

[redsox45]

Great job, Garlon!

It will be interesting to see how this all plays out in the new game.

Will the 162-game issue mean that you have to play 162 game seasons, or will you be able to run 154 game seasons.

Le Grande Orange can correct me on this, but I believe that with the exception of 1903, most seasons since have had 154 games until the past 30 years or so.

How will the Federal League be handled?

Will

[Garlon]

redsox45 - You can play whatever schedule length you like. There are two main reasons why we have adjusted everyone to 162-games: A. Player talent ratings and career progressions will now be more realistic for players in the 1800s primarily. B. With everyone having theoretically played during a 162-game schedule it becomes much easier to set the parameters for when players debut in their first full season. For example, players who had 5000+ ABs debut in their first season of at least 250 AB. If we don't adjust everyone to 162-games what about guys who played in 40 of their teams 80 games and ended up with 175 ABs in their first season? They played half the time but they wouldn't end up debuting that year...this way such a player would now be adjust to 350 ABs for that season and would debut as a rookie.

Since 1962 both NL and AL have played 162-game schduls with the exception of 1072, 1981, 1994, and 1995. I adjusted these years to 162-games.

The Federal League of 1914-1915 will remain in the database. The stats for this league have been adjusted to match the ratios of the AL and NL of 1914 and 1915 respectively. For example, the 1914 Federal League hit .263 and the AL and NL hit a combined .249, therefore all hits in the Federal League were multiplied by .9471 to make the Federal League's batting average also .249 as a whole.