Are AI teams affected by scouting fog of war?

[HH20xx convert]

This may or may not be relevant, but the first iteration of the 'top prospects list', several years ago and before the Baseball America affiliation, specifically stated on the report that it was based upon the ratings of a 'league average scout'. Either the program has changed, or it never was based upon a 'league average scout', or it is not based upon true ratings. Based upon my personal observations in OOTP 11, I am not persuaded this is a true ratings cheat, just my opinion. I look forward to the tests, and perhaps some OOTP team feedback.

[jar2574]

Originally Posted by RonMexico4MVP

By that token, even if the AI were set to not count ratings at all, they'd have no stats to go by, thus defaulting back to the ratings we just altered.

I think the true test would be to make the edits as stated above, but also isolate additional variables like the maturity of the league. In essence, create a quick league, sim long enough for the AI to have some stats to work with (I'd guess at least 5+ years to be safe), then run these tests.

Will it make a difference? Don't know, but I think it would be worth a look.

If we want to isolate whether the AI uses true ratings or scouted ratings then we'll want to eliminate the stat variable. So before I do my inaugural draft I don't plan on simming at all and I plan to turn AI evaluation to 100% ratings.

[SiN8]

Originally Posted by Biggio509

It only took 2 months of simming with AI evaluation set to 100% stats for the AI to have enough stats to change the ranking of prospects by potential. I am really trying not to gloat here as I have been accussed of spreading misinformation and not knowing what I was talking about. What do you have to say now Dr. Dru? I broke the results from that experiment in less than 10 minutes. Ok I guess I will gloat a little.

I don't think Dr. Dru and I are in disagreement with you here. If you change your AI evaluation to 100% stats, then the ratings won't be considered.

The question we are trying to figure out is whether that first weight (ratings) is based on the true ratings or scouted ratings. We've even suggested that if people don't want to give the advantage of true ratings to the AI, then shift the weights more to stats.

[endgame]

I'll respond at this point, time being limited, with a couple of off-the-top-of-my-head comments so take it FWIW. First, thank the good Lord for people like Cras and Biggio, and others I've probably overlooked. Expectations and intentions are far from bretheren, and each subject to their own subjective rules. Yes, even design, however cloaked in a shroud of unknown. Ultimately, we draw not so much our conclusions, but our opinions.

Secondly, taken in part from one of my recent exchanges, I'm growing exceedingly weary of postulations of flaw being substituted for the reality of "it's flawed because no one will describe exactly how it works, therefore it's wrong." A couple of you submitted ideas for examining the game to evaluate contentions and formulate your own opinion about how things 'seem' to work and, utlimately, as a result how you and others would then decide to utilize those functions. That's a good thing. Personal involvement with an eye toward understanding generally is and the degree to which one evangelizes results of their examination separate the caring from the self-indulgent, IMHO. But then there are those, come hither one by one, who will contend they didn't pay X number of dollars to beta-test. So be it.

How does one begin to articulate the idea (hope I'm not alone in this) that formula and detailed mechanics can prove undesireable ends? Trust me, many of us have, over the years pressed Markus for more details about design structure and we still wait. In part, because of the mystery of the game; a part of what makes baseball unpredictable and appreciable. I'm not one to imagine for a minute that if I knew, or Markus shared, every detail about how something worked that I'd still enjoy the game nearly as much. In fact, if he took that approach, you'd have numerous users either 'gaming' the system and feeling cheated themselves, or find it had been emptied of its spirit.

We don't need everything defined and explained. Problems need solved, certainly, but we (Markus) don't owe anyone a 'this is the way this works', only a this is working as designed and this is the intended result, i.e. what it should produce, now measure it with the best tools you have available. Tools, not presumptious pandering of personal propaganda from your pulpit. Alliteration, anyone? Ha!

Remotely, maybe Markus reacts a bit like I do, although the comparison is, admittedly, a stretch. I get frustrated. I try to stay attentive to what may be, potentially, a problem and actually limits or halts gameplay. But honestly, it starts to gnaw at MY enjoyment of the game when it becomes so pervasive! Like a radio station with so much static, I can't enjoy the broadcast.

Call me another crotchety guy trying to have his place.

[jar2574]

Originally Posted by Biggio509

Player development system was disabled on May 1 the current ratings read.

1. Lyles (A. Rodriguez (now ERA over 6) ranked 1 at the beginning of the season
2. Ross Seaton was 4
3. A. Rodriguez
4. D. Keuchel was 4 at the beginning
5. Kyle Greenwalt.

Stats do affect the current rating screen. I disabled player development a bit late 4/21 so current ratings could have changed but stats are definitely affecting the current ratings rankings not just ratings. AI evaluation also has an effect. If switch from 100% stats in AI evaluation then the order is the same at the beginning.

Potential ranking

1. A. Rodriguez
2. Lyles
3. Ross Seaton
4. Kyle Greenwalt
5. Dallas Keuchel

There is no change in potential prospect with 100% ratings on. This is no surprise since there were not stats on day 1 only ratings were used. However, stats did not change anything for potential or league ranking when I went to 100% stats on May 1.

On June 1 stats make a difference in ranking for potential now. Note AI evalution still 100% current stats.

Potential rankings.

1. Bushue
2. Perez
3. Greenwalt
4. Ross Seaton
5. Carlos Quevedo

QED. Player development screen is not based on real ratings and AI evaluation determines how players are ranked. In your test it was only scouted ratings that mattered since you had no stats and the reports did go up with a small change in ratings. Note that still does not say real ratings were used. Only the scouts picked up on a change. Overall might have gone from 79 to 80 instead of 150 to 151 we don't know from the data because we can't see the ratings used for this screen.

QED AI evaluation affects how players are ranked on the player development as the screen says whether you look at from current ratings or potential ratings. League prospect ratings also changed due to current stats by June 1, two months in the season.

Again AI evaluation determines how players are ranked on the PD screen it is not a human cheat!

NB this was done on very low scouting accuracy but it should not matter because AI evaluation was 100% current stats. Ratings were not involved once the stats were enough.

But if the "ratings" portion of the "AI evaluation" is based on "true ratings" and not "scouted ratings" then I think Dr. Dru and SiN8 are correct in one key respect - that the AI doesn't use scouts.

It seems there are two seperate questions: #1 Does the AI use scouts?

#2 Do the player development screen and the top prospect screen rely upon true ratings or AI evaluations (which fluctuate depending upon the weight given to ratings, current year stats, last year's stats and previous years' stats)?

You may have answered question #2 with your tests, but your tests don't seem to answer question #1

Or am I missing something? Just trying to get things clear in my own head.

[SiN8]

Originally Posted by jar2574

But if the "ratings" portion of the "AI evaluation" is based on "true ratings" and not "scouted ratings" then I think Dr. Dru and SiN8 are correct in one key respect - that the AI doesn't use scouts.

It seems there are two seperate questions: #1 Does the AI use scouts?

#2 Do the player development screen and the top prospect screen rely upon true ratings or AI evaluations (which fluctuate depending upon the weight given to ratings, current year stats, last year's stats and previous years' stats)?

You may have answered question #2 with your tests, but your tests don't seem to answer question #1

Or am I missing something? Just trying to get things clear in my own head.

Couldn't have said it any better. Thanks to Baggio for confirming that top prospect and PD page use the AI evaluation. Which in turns uses true ratings for its first weight.

[RonMexico4MVP]

Originally Posted by jar2574

If we want to isolate whether the AI uses true ratings or scouted ratings then we'll want to eliminate the stat variable. So before I do my inaugural draft I don't plan on simming at all and I plan to turn AI evaluation to 100% ratings.

I follow what you're saying, and you may very well find it uses true ratings. The tests will tell.

That being said, the results would thus be appropriate for a league only in the case of setting the AI to value 100% ratings which would without question come with a number of side-effects, like trades/releases/demotions immediately after dropping a single point, etc etc.

If the "ailment" here is the potential cheat by knowing the true rankings, we could alleviate that by including stats in the evaluation and using the AI evaluation to determine the overall ranking. Playing in a "matured" league, like Biggio had done (even for 2 months), would demonstrate that.

Someone else could probably explain it more eloquently than I can, but I hope you get my general idea.

Total aside, I've been playing since OOTP2, and this is the first year I've followed the advice of simming several years then jumping in to play. What a difference. No clue who thought of it first but kudos.

[Dr. dru]

Originally Posted by SiN8

Couldn't have said it any better. Thanks to Baggio for confirming that top prospect and PD page use the AI evaluation. Which in turns uses true ratings for its first weight.

This seems to be the most accurate assement, and I think I understand it. Basically what we have found is:

1) The AI does not use scouts but true ratings
2) The player development screen and top prospect reports is based on ture ratings and stats and the amount is based on what the AI player evaluation is set to.

[Biggio509]

Originally Posted by SiN8

I don't think Dr. Dru and I are in disagreement with you here. If you change your AI evaluation to 100% stats, then the ratings won't be considered.

The question we are trying to figure out is whether that first weight (ratings) is based on the true ratings or scouted ratings. We've even suggested that if people don't want to give the advantage of true ratings to the AI, then shift the weights more to stats.

OOps you caught me before I edited that out. I talked smack too soon. Looking back on the data. It appears that what I saw was two prospects dropping off the list because they were no longer on the list. The edited version shows two things.
1. AI evaluation affects current rating rankings.
2. AI evaluation has no effect on potential rankings.
3. Not in the edit but I noticed in one case league prospect ranking brought a player into the top 5 that was not in the top 5 with potential.

So the scouting used for league prospects, current ratings, and potential, are three different formula. Point 2 should not be surprising because potential ratings are always based on ratings and not stats when your scout is making reports regardless of AI evaluation.

[Dr. dru]

Originally Posted by Biggio509

OOps you caught me before I edited that out. I talked smack too soon. Looking back on the data. It appears that what I saw was two prospects dropping off the list because they were no longer on the list. The edited version shows two things.
1. AI evaluation affects current rating rankings.
2. AI evaluation has no effect on potential rankings.
3. Not in the edit but I noticed in one case league prospect ranking brought a player into the top 5 that was not in the top 5 with potential.

So the scouting used for league prospects, current ratings, and potential, are three different formula. Point 2 should not be surprising because potential ratings are always based on ratings and not stats when your scout is making reports regardless of AI evaluation.

\


So current ratings are based on true current ratings and stats, but potential ratings are just based on the pure potential ratings. That would mean the player development screen is based on true potential ratings correct?

[Dr. dru]

Originally Posted by SiN8

Couldn't have said it any better. Thanks to Baggio for confirming that top prospect and PD page use the AI evaluation. Which in turns uses true ratings for its first weight.

The top prospects page only uses potential ratings though correct? So this is based intirely off of the real ratings.

[Biggio509]

Originally Posted by jar2574

But if the "ratings" portion of the "AI evaluation" is based on "true ratings" and not "scouted ratings" then I think Dr. Dru and SiN8 are correct in one key respect - that the AI doesn't use scouts.

It seems there are two seperate questions: #1 Does the AI use scouts?

#2 Do the player development screen and the top prospect screen rely upon true ratings or AI evaluations (which fluctuate depending upon the weight given to ratings, current year stats, last year's stats and previous years' stats)?

You may have answered question #2 with your tests, but your tests don't seem to answer question #1

Or am I missing something? Just trying to get things clear in my own head.

1. If scouting is on AI teams will hire scouts. There is no way to see how that scout sees players unless you take over the team. Then unless you have AI evaluation turned on you will see current ratings differently than the AI. The AI does not use true ratings for current ratings it uses scouted ratings. Current ratings go into AI evaluation to find overall.
However, potential is more mysterious. We know stats are not used in potential but how the AI or your scout exactly determines potential rating or even current if you don't use AI evaluation is completely unknown and no manual nor Markus has ever given the algorithm of how it is done.
It definitely uses scouted ratings as part of AI evaluation to determine current ratings which are 40% of overall by defualt. It has mysterious unknown formula that weights current stats 40%, last year's stats 15 and two years ago 5 but exactly how the formula converts the numerous offensive ratings, defensive ratings, and stats into a 1 to 80 is a mystery to all but the devs.

2. Is really not answered. All I can say it is uses AI evaluation setting to get current ratings. I assume potential is done the same way your scout would but with better accuracy. The truth is for potential no one but Markus knows if that reflects real ratings or scouted ratings that are better than your scout's.

It is clear what ever method used gives a better outcome to the scouting than your scout when it ranks by potential. League prospect ranking is also a mysterious formula I have no idea about. It is definitely different from potential and current but I am not sure if it is different scouts with different abilities than those who do potential or a different formula. The OOTP 11 manual was not clear on what scouts if any were used for BA top prospect rankings.

I think part of this discussion gets confusing when you try to talk about it. The claim Dr. Dru makes which I think is not correct is that true ratings are used and not scouting. I doubt that because I have seen the AI make sure trades that are bad meaning it valued what my scout was a 1 star who never made the majors for what my scout thought was a 5 star who did make the majors. I have seen it pass up 5 star players who look like 1 stars in the draft and FA pool. In 11 I signed a guy who had been sitting in the FA pool for 2 years after the inaugural draft that my scout said was a 1 star and the AI would not touch. He was meant to be a pule on my SA team and became a solid major league leftfielder. So I don't believe the AI gets a magical view of the player's true potential.

The problem here is scouting is different for potential and current ratings. Current ratings definitely use AI evaluation. Potential uses another method. I have no idea how the AI weighs potential versus current in trades. All I know is there is more weight on potential if the AI favors prospects. So if the AI uses real ratings for potential but AI evaluation for current it can sway a trade or maybe current is bad enough to kill it. Depends on trade settings.

The only really clear thing is that the AI calculates current overall according to AI evaluation. How it mixes that with potential in determining trades is somewhat mysterious but affected by favor prospects or favor vets. I think favor prospects weighs potential (mystery how it is formed) heavier while favor vets using AI evaluation more heavily.

[Dr. dru]

Originally Posted by Biggio509

1. If scouting is on AI teams will hire scouts. There is no way to see how that scout sees players unless you take over the team. Then unless you have AI evaluation turned on you will see current ratings differently than the AI. The AI does not use true ratings for current ratings it uses scouted ratings. Current ratings go into AI evaluation to find overall.
However, potential is more mysterious. We know stats are not used in potential but how the AI or your scout exactly determines potential rating or even current if you don't use AI evaluation is completely unknown and no manual nor Markus has ever given the algorithm of how it is done.
It definitely uses scouted ratings as part of AI evaluation to determine current ratings which are 40% of overall by defualt. It has mysterious unknown formula that weights current stats 40%, last year's stats 15 and two years ago 5 but exactly how the formula converts the numerous offensive ratings, defensive ratings, and stats into a 1 to 80 is a mystery to all but the devs.

2. Is really not answered. All I can say it is uses AI evaluation setting to get current ratings. I assume potential is done the same way your scout would but with better accuracy. The truth is for potential no one but Markus knows if that reflects real ratings or scouted ratings that are better than your scout's.

It is clear what ever method used gives a better outcome to the scouting than your scout when it ranks by potential. League prospect ranking is also a mysterious formula I have no idea about. It is definitely different from potential and current but I am not sure if it is different scouts with different abilities than those who do potential or a different formula. The OOTP 11 manual was not clear on what scouts if any were used for BA top prospect rankings.

I think part of this discussion gets confusing when you try to talk about it. The claim Dr. Dru makes which I think is not correct is that true ratings are used and not scouting. I doubt that because I have seen the AI make sure trades that are bad meaning it valued what my scout was a 1 star who never made the majors for what my scout thought was a 5 star who did make the majors. I have seen it pass up 5 star players who look like 1 stars in the draft and FA pool. In 11 I signed a guy who had been sitting in the FA pool for 2 years after the inaugural draft that my scout said was a 1 star and the AI would not touch. He was meant to be a pule on my SA team and became a solid major league leftfielder. So I don't believe the AI gets a magical view of the player's true potential.

The problem here is scouting is different for potential and current ratings. Current ratings definitely use AI evaluation. Potential uses another method. I have no idea how the AI weighs potential versus current in trades. All I know is there is more weight on potential if the AI favors prospects. So if the AI uses real ratings for potential but AI evaluation for current it can sway a trade or maybe current is bad enough to kill it. Depends on trade settings.

The only really clear thing is that the AI calculates current overall according to AI evaluation. How it mixes that with potential in determining trades is somewhat mysterious but affected by favor prospects or favor vets. I think favor prospects weighs potential (mystery how it is formed) heavier while favor vets using AI evaluation more heavily.

I think for purposes of trade the AI does not use scouts but also does not nesisarily use real ratings. I think for trades the AI to a certain extent has some randomness to it. I agree that mysterious might be the best way to put it.

[Biggio509]

Originally Posted by Dr. dru

\


So current ratings are based on true current ratings and stats, but potential ratings are just based on the pure potential ratings. That would mean the player development screen is based on true potential ratings correct?

No overall rating is where the player is currently compared to players at his position in the league. It is determined by scouted ratings and stats. AI evaluation affects Overall ratings for the AI and you if use it. Potential ratings are a mystery to how they are formed.

For your scout it is based on true potential and how accurate the scout is. So if you spend the max on scout and have a legendary scout a player who has gone from rookie to AAA might be scouted accurately enough to where scouted ratings = true ratings but not a greenhorn in rookie who just joined the org. I assume that the AI calculates potential in the same your scout does that is a blurred version of true ratings with an error component but there is no documentation to say how potential ratings are formed. It is a mystery. I assume the top prospects list and player development use the same information settings and are much closer to true ratings than your scout usually is but I really have no idea how the AI calculates the overall potential.

[jar2574]

Originally Posted by Biggio509

It is clear what ever method used gives a better outcome to the scouting than your scout when it ranks by potential. League prospect ranking is also a mysterious formula I have no idea about. It is definitely different from potential and current but I am not sure if it is different scouts with different abilities than those who do potential or a different formula. The OOTP 11 manual was not clear on what scouts if any were used for BA top prospect rankings.

After frustration with a couple drafts in OOTP 11, I went in and looked at a couple guys on the BA top prospect list using the player editor. My recollection is that they were ranked in the list according to their true potential rankings. Someone could do the same thing easily for OOTP 12.

Anyway, for good or for ill, now after a draft I check the BA rankings to see how I did.

[jar2574]

Originally Posted by Biggio509

I think part of this discussion gets confusing when you try to talk about it. The claim Dr. Dru makes which I think is not correct is that true ratings are used and not scouting. I doubt that because I have seen the AI make sure trades that are bad meaning it valued what my scout was a 1 star who never made the majors for what my scout thought was a 5 star who did make the majors. I have seen it pass up 5 star players who look like 1 stars in the draft and FA pool. In 11 I signed a guy who had been sitting in the FA pool for 2 years after the inaugural draft that my scout said was a 1 star and the AI would not touch. He was meant to be a pule on my SA team and became a solid major league leftfielder. So I don't believe the AI gets a magical view of the player's true potential.

I agreed with you before SiN8's tests. But after those tests, I think the AI in OOTP 12 likely gets a view at true ratings before a trade.

That said, I think you'll still be able to get a good trade from the AI because the AI is also looking at stats.

If stats are 60% of the AI evaluation, then the AI may give away a future superstar if he's struggled recently on the field.

So when I look at the "top prospect" screen (which is based on true potential) and simply go from #1 down through #100 until I get a good deal, what is actually happening is that I am waiting until the AI undervalues someone with high potential who has struggled recently on the field lately. And I get them for cheap.

This explains a lot of my success in OOTP11 and may be tough for me to avoid doing in OOTP12. (Because one of my personal flaws is that I always want to win.)

[Dr. dru]

Originally Posted by jar2574

I agreed with you before SiN8's tests. But after those tests, I think the AI in OOTP 12 likely gets a view at true ratings before a trade.

That said, I think you'll still be able to get a good trade from the AI because the AI is also looking at stats.

If stats are 60% of the AI evaluation, then the AI may give away a future superstar if he's struggled on the field.

So when I look at the "top prospect" screen (which is based on true potential) and simply go from #1 down through #100 until I get a good deal, what actually happens is that I wait for someone with high potential who has struggled on the field lately. And I get them for cheap.

This explains a lot of my success in OOTP11 and may be tough for me to avoid doing in OOTP12. (Because one of my personal flaws is that I always want to win.)

Not a flaw to want to win. The way I play I will make trades or sign free agents to the poin that my team generally wins 90 -120 games a year, makes it to the world series about half the time, and wins a Championship about 3 times every ten years or so. Losing isn't that fun.

[Biggio509]

Originally Posted by jar2574

If stats are 60% of the AI evaluation, then the AI may give away a future superstar if he's struggled recently on the field.

It is not that simple and can not be that simple. AI evaluation is used for overall which reflects current ability. Your trade settings at neutral, favor prospects, or favor vets affect the ratings of current versus potential in some way that is not editable and not really known. In your example the AI still may not give away the superstar who has struggled lately even if you have current stats sets to 100% in AI evaluation. That only affects how the AI looks at today's overall rating not the player's potential or how much the AI values current over potential or potential over current. The AI will somehow balance current against potential. If you have favor prospects selected the AI may still go with the high potential guy with a crappy current overall. If you have favor vets and AI evaluation set to 100% current stats then overall is probably weighed more heavily and you will have a hard time trading a guy who is performing poorly at the moment even if he has high potential.

What I can say is by defualt the game uses current scouted ratings and stats to get the current overall. The mystery is how potential is gathered. It is based on ratings but it is a still a mystery as to say what % power plays in overall or potential rating versus say eye or defensive stats. It is also unknown what the weight is for potential and current (AI evaluation). At neutral overall may be 50% and potential 50%. At favor prospects overall may 20% or 30% percent and potential have a 80% or 70% weight. That is part of the trade AI engine we don't see. We can affect it by the option but we have no idea how the AI weights current ratings versus potential and exactly how much favoring prospects or favoring vets changes that. Also I am not sure what choosing say hard trade AI does to the evaluation of current and potential used in a trade.

[jar2574]

I tried to test whether the AI uses scouts in drafts. Unfortunately, the results of my testing were inconclusive.

I took a brand new 10-team league with no history and deleted all the players. Then I created 10 players and turned them into 18-year old catchers with current and potential ratings at 250, 249, 248, etc., respectively, and with experience at 200 for catching and 0 for every other position.

I gave them all max in personality traits and made them all 1's at every injury-rating. They all weigh the same and throw and bat right-handed.

I set player ratings to 100% of evaluation.

I then ran an inaugural draft 6 times, 3 times with scouting and 3 times with scouting off.

The 10 players were taken in the exact same order every time, which would seem to indicate that scouts do not matter. But the order in which they were taken was surprising - 248, 249, 242, 244, 247, 250, 241, 246, 245, and 243. (They were named with the ratings given to them.)

So on the one hand, scouted and scout-less drafts happened in the same order, with random teams in different slots. But on the other hand, I can't figure out why the players went in that order. The only thing I can think of is that (1) I missed a variable in the player editor (not makeup, not handedness, not height or weight, not G/F ratio, not stamina, not velocity, not Hold, and not current and potential ratings, but something else.) Or (2) each player has an internal development engine and will age and progress differently, and that development engine cannot be edited in the player editor but has an effect on AI evaluation. Or (3) something else I'm not smart enough to think of.

Interestingly, the player development report had these guys listed in the same order.

That's enough testing for me for one day. Some mysteries of the game will remain mysteries.

[SiN8]

Originally Posted by jar2574

I tried to test whether the AI uses scouts in drafts. Unfortunately, the results of my testing were inconclusive.

I took a brand new 10-team league with no history and deleted all the players. Then I created 10 players and turned them into 18-year old catchers with current and potential ratings at 250, 249, 248, etc., respectively, and with experience at 200 for catching and 0 for every other position.

I gave them all max in personality traits and made them all 1's at every injury-rating. They all weigh the same and throw and bat right-handed.

I set player ratings to 100% of evaluation.

I then ran an inaugural draft 6 times, 3 times with scouting and 3 times with scouting off.

The 10 players were taken in the exact same order every time, which would seem to indicate that scouts do not matter. But the order in which they were taken was surprising - 248, 249, 242, 244, 247, 250, 241, 246, 245, and 243. (They were named with the ratings given to them.)

So on the one hand, scouted and scout-less drafts happened in the same order, with random teams in different slots. But on the other hand, I can't figure out why the players went in that order. The only thing I can think of is that (1) I missed a variable in the player editor (not makeup, not handedness, not height or weight, not G/F ratio, not stamina, not velocity, not Hold, and not current and potential ratings, but something else.) Or (2) each player has an internal development engine and will age and progress differently, and that development engine cannot be edited in the player editor but has an effect on AI evaluation. Or (3) something else I'm not smart enough to think of.

Interestingly, the player development report had these guys listed in the same order.

That's enough testing for me for one day. Some mysteries of the game will remain mysteries.

Great job on this experiment! Perhaps the cutoff ceiling is 200 and anything above is not accounted for. You might want to retry with players 190 to 200.