Japanese roster/historical database project

[murmur13]

Hi,

This post is for the NPB historical Lahman database project, for OOTP.

***The most recent version of the historical database is below***

NPB Historical Database, version 0.81 (6/17/07)

The database contains the following:

1) A historical database, from 1936 to the present, for Nippon Professional Baseball. It has complete batting and pitching stats for the majority of historical players, and partial fielding stats as well.

2) The same historical database, with MLB equivalencies for all NPB seasons, along with straight MLB stats for anyone w/MLB experience that played in NPB. These MLEs were culled from Jim Albright's Japanese baseball page (http://baseballguru.com/jalbright/). These MLEs will make it really easy to import any NPB player into your MLB universe, if you choose to do so.

3) Two separate databases for rookie import (after starting a historical game). The first DB is the NPB database w/MLB equivalencies, and the second is a modified Lahman database. The difference is that this NPB database ONLY has players who begun major league play in Japan, and the MLB database only has players who begun major league play in the States. So for example, Hideki Matsui and Ichiro are only in the modified NPB database, whereas foreign Japanese starts (e.g. Alex Cabrera, Petagine, etc.) are only in the modified MLB database. This is setup this way so that rookies can be first imported where they started their career, and then develop and move based on the historical replay.

Detailed instructions for using this database are in the readme.txt file of the database.

*** Update: 7/1/08 ***

I am currently working on an update to the Nippon Professional Baseball historical database (in Lahman format) that can be imported into OOTP. I've tested it, and the following changes have been added:

* Full statistics for the 2007 season

* Updated franchise information, making franchise moves and expansion/contraction possible

* A separate database with MLB equivalencies for NPB players based on historical era (new!), making for more realistic statistics when playing concurrent historical MLB and NPB seasons

* Accurate and working importing of award winners and hall of famers

This is in addition to the features from last year's version of the database, which is a detailed Lahman database for Japanese baseball with every NPB player, complete batting and pitching statistics for the majority of players, fielding statistics from 1990-now and for select earlier players, all stars, award winners, hall of famers, team history, franchise history, etc.

I would release it now, but the database has the following flaws, which were in last year's version:

* Missing secondary batting stats (doubles, triples, caught stealing) for some players from the 30s to the 80s

* Missing fielding position information for some players from the 30s to the 80s

* Lots of missing fielding statistics from the 30s to the 80s (current workaround: fake stats based on an average NPB player)

The last issue won't be fixable for now, as fielding statistics before the 80s are extremely difficult to find.

However, the Data Warehouse at Japan Baseball Daily has been updated since I last posted this database, and currently contains nearly all of the missing batting statistics and positional information for nearly every player that has played in NPB. This is great, except that it is not in database/spreadsheet format. Instead it is a bunch of tables on separate web pages on the site. I would love to add this information, but it would probably take months to enter it all manually into the NPB Lahman database myself.

Fortunately, it looks like the data at Japan Baseball Daily is in a pretty standard format, and could be pulled from the site into a spreadsheet/database using a script. Unfortunately, I don't have the expertise to do this. However, if there is anyone that does and is willing to help, please let me know. I would love to have a more complete version of the database available for historical players. If you can help with this project, or if you are aware of a newer NPB database that has this information, please PM me.

[Erich Ingram]

That sounds awesome

[DRaysGoRocco]

This is excellent, I can't wait to try it out. I can't answer #1 or #2 as I've tried to find both, but #3 really won't be a problem, just send it to ootpmodsquad or padrefan's site.

[PadresFan]

Originally Posted by murmur13

3) Does anyone want to host this when it's done?

I'll host!

[mh2365]

It would be great if you could use this at the same time as the Lahman db .. is that possible?

[Claymore]

This sounds really, really cool!

[murmur13]

MH,

This is supposed to be used with the Lahman database...in fact, this will include a modified Lahman DB (with converted MLEs for NPB players with MLB experience) so that you can run both leagues at once.

I'll finish this up, with partial fielding stats and a warning to turn off expansion for NPB. I'll try and finish it up tonight or tomorrow.

Also, are there any Japanese facepacks around? I've found some, but they don't seem to actually have any actual player names.

[kagnew35]

Check with Fidel Montoya for Japanese stats and possible photo paks, he lives in Japan.

allsimbaseball.com and email him

[CONN CHRIS]

This sounds fantastic!

Have you run any pre-war tests? I am curious to know how Victor Starffin performed.

[murmur13]

Nope...no pre-war tests. Even though there's data from 1937 and beyond, I've been having problems importing the teams correctly during the one league era of Japanese baseball. I think the game might be hardcoded to expect two leagues, and messes up if it doesn't get that. Though someone could use this DB to import Starrfin into their own universe, until someone figures out how to get this to work.

BTW, I'm almost done. I'm trying to clean up a few errors, and hope to have this posted sometime tomorrow (Wed) evening.

[CONN CHRIS]

Originally Posted by murmur13

Nope...no pre-war tests. Even though there's data from 1937 and beyond, I've been having problems importing the teams correctly during the one league era of Japanese baseball. I think the game might be hardcoded to expect two leagues, and messes up if it doesn't get that. Though someone could use this DB to import Starrfin into their own universe, until someone figures out how to get this to work.

BTW, I'm almost done. I'm trying to clean up a few errors, and hope to have this posted sometime tomorrow (Wed) evening.

Are the years that they played two 'half' seasons combined into single seasons? I'd imagine that could make importing a bit hairy.

[mgom27]

Going to be adding Real Personal and Real Minor League Team Rosters to?

[murmur13]

Chris,

That could be hairy...but the more hairy thing is that the game seems to be hard-coded to expect two leagues. So during the one league period (from 1937-1949), you can import and it will put all the teams in one league. But then the game wont start because of there being an uneven number of teams in each "league" (8 in one league, none in the other). In fact, I would recommend starting any historical sim (at the earliest) in 1958. Many of the seasons from 1950-1957 have uneven teams in the Central and Pacific, and the historical import engine seems to hate uneven teams.

Mgom,

The players are all real players. I have a few sources of ni-gun (Japanese AAA) stats, but I haven't included them for a variety of reasons. It might be something to add for the future...but I just want to get this out first.

[CONN CHRIS]

A 1937 fictional league with real team names and a draft might do the trick. If the pre-war data is in the DB I may fiddle with that down the line.

About three years ago when I played Puresim (I have limited experience with OOTP) I edited (poorly) my own Lehman DB to include pre war Nippon and Negro League players. I had one fictional major league throughout so all the players were imported into the same league draft every year. I only got up to the mid forties (from 1871) before I abandoned Puresim for TBCB. (Nothing to do with Pursim, which I loved. I just happen to be a crazy boxing fan.)

I really look forward to re-kindling my interest in Japanese Baseball with this.

Thank you!

[legendsport]

Originally Posted by murmur13

Chris,

That could be hairy...but the more hairy thing is that the game seems to be hard-coded to expect two leagues. So during the one league period (from 1937-1949), you can import and it will put all the teams in one league. But then the game wont start because of there being an uneven number of teams in each "league" (8 in one league, none in the other). In fact, I would recommend starting any historical sim (at the earliest) in 1958. Many of the seasons from 1950-1957 have uneven teams in the Central and Pacific, and the historical import engine seems to hate uneven teams.

Mgom,

The players are all real players. I have a few sources of ni-gun (Japanese AAA) stats, but I haven't included them for a variety of reasons. It might be something to add for the future...but I just want to get this out first.

I've been working on a Japanese historical db as well, though it's nowhere near as close to completion as murmur's is. What I did for the 1936-38 half seasons was put the Summer season as 'stint' 1 in the Lahman-style db and the Fall season as 'stint' 2. That way, on import, the sim looks at it as one complete season, but you don't need to worry about combining the stats yourself.

My experience (I've got 1936-57 or so done, db-wise) has been that you have to basically manually set up the Japanese leagues because OOTP is set up for MLB and NPB and MLB have never really meshed because of odd-numbers of teams, the single-league setup of the 30s & 40s and so on. You can get around the odd number of teams problem (which is a schedule problem, the sim CAN handle an odd number of teams, it just can't schedule a league with an odd number of teams). So you'll have to do some workarounds, but it is possible to simulate the actual NPB setups of the 30s-50s.

[murmur13]

The databases are up here. Padresfan, if you want to host these, feel free. And if anyone has any questions, comments, or corrections, feel free to post in this thread or PM me. The contents of the readme file are pasted below. Enjoy!

----------------

The following folders contain various databases for OOTP 2007 that will allow you to import Japanese players, or play historical seasons in Nippon Professional Baseball (NPB), either as a stand-alone league or in concert with a historical MLB. First, I'll describe what's in these folders. (But if you want to go straight to the directions on how to use this, scroll down to INSTRUCTIONS FOR RUNNING CONCURRENT NPB AND MLB HISTORICAL SEASONS.)

1_Raw_NPB

This folder contains the actual NPB stats that I've been able to find, starting with the first professional season in Japanese baseball (1937) and continuing to the 2006 season. Batting and pitching statistics are fairly intact. However, there are huge gaps in what I've been able to find for fielding statistics (as things are extremely spotty pre-1992). In order to deal with this, players have dummy fielding entries with no fielding stats.

These players also do not have positional information that I could glean from other databases. So famous players from the 30s to approximately 1990 have correct positions. However, lesser known players have essentially a random position. (If you leave position blank, the game seems to import everyone as a first basemen). If anyone has NPB fielding stats that are not in this database, please PM me (murmur13) at the OOTP forums.

2_NPBwMLBeq_Start

This folder contains two folders: NPB and MLB. The NPB folder contains the same information as in 1_Raw_NPB, with the following additions:

1) NPB batting and pitching statistics have been converted to thieir major league equivalencies (MLEs), based on Jim Albright's NPB to MLB conversion factors. Articles describing how these factors were derived can be found for...

Batting: http://baseballguru.com/jalbright/an...lbright19.html
Pitching: http://baseballguru.com/jalbright/an...lbright26.html

For reference, here are the adjustment factors used:

Batting:
Hits .932
Doubles .980
Triples 1.577
Homers .604
Walks .805

Pitching:
Hits 1.076
Homers 1.236
Walks 1.072
Strikeouts .884

2) MLB stats for any player that has appeared in NPB. For example, for Ichiro Suzuki, this database has his MLEs for the Orix BlueWave, and his stats for the Mariners.

The MLB folder contains the Lahman MLB database, along with the converted NPB statistics for anyone in NPB that has played in MLB.

So, both the MLB and NPB folders have full career stats for players that have played in both NPB and MLB. The MLB folder also has players that have ONLY plyaed in MLB, and the NPB folder has players that have ONLY played in NPB.

This folder can easily be used to start a historical, concurrent NPB and MLB. However, each database has an Ichiro, a Matsui, a Petagine, etc. So if you use these databases to import rookies, then there will be multiples of any MLB players that have played in NPB.

3_NPBwMLBeq_Import

This folder is to be used to import rookies in a historical league. The data in these two databases (NPB and MLB) are exactly the same as in 2_NPBwMLB_Start. However, all career statistics for Japanese players or foreign players that ONLY played in NPB are in the NPB folder. Any non-Japanese player with MLB experience has his full stats in the MLB folder. This will allow, for example, Ichiro to be available in the Japanese draft (but not the MLB draft), and players like Petagine, Tuffy Woods, Cabrera to be available in the MLB draft.

INSTRUCTIONS FOR RUNNING CONCURRENT NPB AND MLB HISTORICAL SEASONS

1. Click on "Add historical league", and select the master.csv file in \2_NPBwMLBeq_Start\NPB. NOTE that it may be problematic to start earlier than 1958. From 1937-1949, NPB was one league, and from 1950-1957, there were uneven teams in each league. The default import has problems with these setups. (However, if anyone knows a fix/workaround, PM me and I'll include it in this file.)

Go to the structure tab. When importing the league, it will import as an American (USA) league. Change the league nationality to Japan. However, when you do this, it will change team cities, so you might want to remember the cities/company names in order to reset them correctly. (Also note that the team names will NOT automatically update as you play...so for example, the Taiyo Whales will remain as the Taiyo Whales throughout history and never change to the Yokohama Bay Stars unless you do it manually.)

Go to the Names and Affiliations tab. Note that the league names are AL and NL. This is because the game WILL NOT import teams unless they're AL or NL. So change the league names: AL is the Pacific League (DH), NL is the Central League (no DH). (Also, the Japanese version of the Cy Young Award is the Sawamura Award...all other award names are the same).

Go to the Rules tab. Please make sure to DISABLE "automatically expand league". So far, expansion DOES NOT work with this database. Again, future releases may have instructions on how to fix this.

If you want to setup foreign player limits, you can (NPB is 4 players). However, note two things. First, you MUST specify the foreign pitcher and hitter limit. If you don't set this, it will ignore the player limits. Second, AI teams will sometimes get around this by putting extra foreign players on their reserve or AAA squads. So far, I am not aware of any current fix for this.

2. Click on "Add historical league, and select the master.csv file in \2_NPBwMLBeq_Start\MLB.

3. Tweak settings as you'd like (finances, foreign player limits, etc.).

4. Start game.

5. If you're interested in importing historical rookies...
5a. Go to Game setup. Then click on the League Setup tab. Choose your NPB league. Click on the rules tab. Go down to Historical Rookies Options, and set the following path:

your directory\NPB_DB\3_NPBwMLBeq_Import\NPB

5b. Do the same as above for your MLB league. Set the following path:

your directory\NPB_DB\3_NPBwMLBeq_Import\MLB

6. The final step. When starting the game, any player that has played in both MLB and NPB will appear in BOTH leagues. So search for any duplicates (usually do a search for foreigners on NPB teams, and then do a search on that player name), and delete the duplicate that is NOT on a roster.

That's it. Enjoy. If you have any questions, please post in the OOTP forums.

[Gambo]

This is an awesome project, this coming from someone who does DBs!

I'd love to combine this to some extent to work with my database. I would at least love to get the guys who began their career in Japan such as Ichiro etc and move their MLEs into my DB.

[murmur13]

Gambo,

Attached are the MLB equivalencies for NPB seasons for players that have played in both NPB and MLB. This includes both Japanese in MLB (Ichiro) and MLBers that have gone to Japan (Warren Cromartie, Robert Petagine, Alex Cabrera, etc). IDs are the same as in the Lahman DB.

[Gambo]

Awesome!

[pro4]

Anyone have the uniforms for the Nippon league and their minor league teams??? This is a awesome set! Who cares about the 2007 mlb rosters!