Study #1 - What is clutch worth in OOTP4?

Jason Moyer · 05-22-2002, 09:52 AM

I created a 2-team test league, both teams exactly the same, and ran a series of tests to determine the effects of clutch. Here are the results. In all of these tests, Team B is exactly like Team A with all players having their clutch rating set to "normal". The record after 10 sims is in parenthesis:

Team A Normal - 81.5 Wins (5-4-1)
Team A Great - 84.6 Wins (4-5-1)
Team A Suffers - 75.2 Wins (1-9-0)

The conclusion? Suffering in clutch situations has more of an impact than being great in clutch situations.

Being great in the clutch, based on this study, is worth about 3-4 wins a year. However, notice the sub-.500 win/loss record. My conclusion is that there really is no big advantage to having a great clutch rating over a normal clutch rating.

Suffering in the clutch is a different story. Based on this study, a team with all poor clutch performers would have 5-6 extra losses a year against normal competition. Even more telling is the win loss record of 1-9. My conclusion? Don't field teams of 25 guys who suffer in clutch situations.

Unless, of course, they're all superstars.

Overall, based on this study, I would say that clutch ratings have some slight effect, primarily negative, but in practical terms, when choosing whom to sign or put in your lineup, I would use it only as a tiebreaker between 2 players with exactly the same abilities.

To conclude, clutch hitting may exist, but if it does its effects are negligible.

Jason

[ 05-22-2002, 06:16 PM: Message edited by: Jason Moyer ]

Yyzlin · 05-22-2002, 12:52 PM

Very interesting study. I used to always value GREAT clutch ratings over Normal clutch very highly. But I guess this study proves that the difference isn't that great. I have always avoided players with Suffer cluth rating. I would imagine that the CLUTCH rating would become much more apparent during the playoffs though. It would be interested to see how those 3 teams fare in a playoff. I think the GREAT team would have a great advantage over normal in the playoffs than in the regular season.

JAttractive · 05-22-2002, 12:58 PM

Is their a scale of Clutch-Suffer or is this a yes-no type thing? I have a couple players who are listed as normal but every playoffs they hit a ton of homeruns and their average is about .100 points higher. I was sure they were "nearly" clutch or something.

I have noticed a huge difference in the playoffs though in my players listed as clutch vs. suffers, however I haven't played enough games to be accurate nor do I play many suffers-types so it is not really a fair comparison. Still I am seeing a trend in my limited games.

Jason Moyer · 05-22-2002, 12:59 PM

On an individual basis, I would estimate that clutch really doesn't mean much. I'll run a study on the impact that having a single clutch hitter has and another on how much a single clutch pitcher has.

Taken over an entire team tho, I think I should adjust my conclusions. Clearly it's not irrelevant. The difference between a team of 25 players with GREAT clutch ratings and a team with Suffers clutch ratings in my study is 9.4 wins per season. That is a pretty significant number.

Jason

JAttractive · 05-22-2002, 01:01 PM

Oh and I notice the AI almost always brings in my Clutch reliever in the playoffs for important situations (I tend to let the AI handle pitching until late in the game), even though I have better pitchers on the bench that it went to all season. This guy is used nowhere near as much during the regular season as he is in the playoffs. Again this could be due to limited testings/games but it is possible their is a significant enough advantage to using them in playoffs.

Sony · 05-22-2002, 01:01 PM

I hope you turned injuries off

Jason Moyer · 05-22-2002, 01:12 PM

Also, FWIW, I'm running some more studies now to demonstrate:

1. The impact of A vs E defense at each position.
2. The impact of a single player who is twice as good as the league at all hitting ratings
3. The impact of a single player who is half as good as the league average at all hitting ratings

My final report will be a guideline on building the ultimate compromise lineup, assuming that each player can only be good at one thing (either A/A fielding with half of average hitting or E/E fielding with twice the average hitting), with some studies on how each of my compromise ideas would actually perform on the field. I'll probably do some studies on the impact a twice-average pitcher in each role has, and the impact of speed. Then I can do another summary on what has the biggest impact in the game.

Jason

Jason Moyer · 05-22-2002, 01:16 PM

<blockquote>quote:<hr />Originally posted by Sony:
I hope you turned injuries off<hr /></blockquote>I'm using the following settings:

replay mode (no player dev)
injuries off
trades off
endurance off

Basically I just want to create a completely even playing field, so I can tinker with a few ratings and see what sort of overall impact they have.

Something interesting from the defensive study I'm doing now - all things being equal, a SS or 2B with A range is worth 10 more wins per season than a SS or 2B with E range. Considering that an entire team of GREAT clutch performers is only worth 10 wins over a team of Suffers clutch performers, that is a lot of bang for the buck.

Jason

Carplos · 05-22-2002, 05:53 PM

<blockquote>quote:<hr />Originally posted by Jason Moyer:
Also, FWIW, I'm running some more studies now to demonstrate:

1. The impact of A vs E defense at each position.
2. The impact of a single player who is twice as good as the league at all hitting ratings
3. The impact of a single player who is half as good as the league average at all hitting ratings

My final report will be a guideline on building the ultimate compromise lineup, assuming that each player can only be good at one thing (either A/A fielding with half of average hitting or E/E fielding with twice the average hitting), with some studies on how each of my compromise ideas would actually perform on the field. I'll probably do some studies on the impact a twice-average pitcher in each role has, and the impact of speed. Then I can do another summary on what has the biggest impact in the game.

Jason<hr /></blockquote>That takes a lot of the fun out of the game..

Jason Moyer · 05-22-2002, 06:59 PM

Howso?

From what I can tell, everything I've said is stating the obvious and applies to real life.

1. The defensive study just basically reaffirms the defensive weights Bill James uses, with the catcher being a notable exception, altho a good quality catcher being rare has more to do with the athletic requirements for the position rather than the importance of having a good defensive catcher.

2. A study I haven't published yet shows that A steal/A speed guys aren't really worth anything in terms of producing wins. Again, Bill James has been saying this for 20 years about real baseball, so no big surprises.

3. If I do a study comparing the impact of starters, relievers, and closers, and it shows that the starters have the most impact, followed by the middle relievers, followed by the closers, it would just show that the game is pretty close to reality.

The stuff I'm been testing so far is pretty obvious imho. I'm actually surprised at how close the game mirrors real life in some respects.

Jason

William4192 · 11-26-2003, 12:19 PM

Great Thread!

Gastric ReFlux · 11-26-2003, 12:21 PM

This thread is clutch.

jaxmagicman · 11-26-2003, 12:24 PM

Can we try this study with OOTP5?

sporr · 11-26-2003, 12:28 PM

Quote:

Originally posted by Jason Moyer
On an individual basis, I would estimate that clutch really doesn't mean much. I'll run a study on the impact that having a single clutch hitter has and another on how much a single clutch pitcher has.

Taken over an entire team tho, I think I should adjust my conclusions. Clearly it's not irrelevant. The difference between a team of 25 players with GREAT clutch ratings and a team with Suffers clutch ratings in my study is 9.4 wins per season. That is a pretty significant number.

Jason

This is my thought as well. Unless you had an entire team of clutch players, it's not really going to make that much of a difference. However, it isn't going to hurt anything to get guys who are clutch over guys that aren't.

fat dude · 11-26-2003, 12:46 PM

Am I understanding you correctly when I read 10 games to mean you only simmed 10 games to come to your conclusions? I hardly think 10 games is enough to prove a thing. I think you're on the right track, but I would need to see a whole season simmed before I made any conlusions. Maybe I'm reading this wrong though.

sporr · 11-26-2003, 12:56 PM

Quote:

Originally posted by fat dude
Am I understanding you correctly when I read 10 games to mean you only simmed 10 games to come to your conclusions? I hardly think 10 games is enough to prove a thing. I think you're on the right track, but I would need to see a whole season simmed before I made any conlusions. Maybe I'm reading this wrong though.

I took him to mean that he simmed 10 seasons.

Muppetus Galacticus · 11-26-2003, 01:38 PM

Try it with a team of Ross Gloads.

Muppetus Galacticus

ssmarsh · 11-26-2003, 01:55 PM

I took it to mean 10 games.

Henry · 11-26-2003, 02:04 PM

It looks like 10 games given the win-loss records. I would think at least 2-3 seasons would be necessary.

jaxmagicman · 11-26-2003, 02:10 PM

Quote:

Originally posted by Henry
It looks like 10 games given the win-loss records. I would think at least 2-3 seasons would be necessary.

I think it was seasons. The win-loss records were seasons over .500 as wins, and season under .500 as losses and the ties were seasons at .500.

05-22-2002, 12:52 PM	#2
Yyzlin All Star Reserve Join Date: Apr 2002 Location: The Triangle Favorite Band: Iron Maiden Posts: 759	Very interesting study. I used to always value GREAT clutch ratings over Normal clutch very highly. But I guess this study proves that the difference isn't that great. I have always avoided players with Suffer cluth rating. I would imagine that the CLUTCH rating would become much more apparent during the playoffs though. It would be interested to see how those 3 teams fare in a playoff. I think the GREAT team would have a great advantage over normal in the playoffs than in the regular season. __________________

05-22-2002, 12:59 PM	#4
Jason Moyer Hall Of Famer Join Date: Feb 2002 Location: Pittsburgh Posts: 5,003	On an individual basis, I would estimate that clutch really doesn't mean much. I'll run a study on the impact that having a single clutch hitter has and another on how much a single clutch pitcher has. Taken over an entire team tho, I think I should adjust my conclusions. Clearly it's not irrelevant. The difference between a team of 25 players with GREAT clutch ratings and a team with Suffers clutch ratings in my study is 9.4 wins per season. That is a pretty significant number. Jason __________________ "I pretty much popped everything cold turkey. We were doing steroids they wouldn't give to horses." -- Tom House "I was very fortunate to have a pitching coach by the name of Tom House...Tom, I really miss those days that we spent in the weight room and out on the field working together." -- Nolan Ryan's HoF Induction Speech

05-22-2002, 01:12 PM	#7
Jason Moyer Hall Of Famer Join Date: Feb 2002 Location: Pittsburgh Posts: 5,003	Also, FWIW, I'm running some more studies now to demonstrate: 1. The impact of A vs E defense at each position. 2. The impact of a single player who is twice as good as the league at all hitting ratings 3. The impact of a single player who is half as good as the league average at all hitting ratings My final report will be a guideline on building the ultimate compromise lineup, assuming that each player can only be good at one thing (either A/A fielding with half of average hitting or E/E fielding with twice the average hitting), with some studies on how each of my compromise ideas would actually perform on the field. I'll probably do some studies on the impact a twice-average pitcher in each role has, and the impact of speed. Then I can do another summary on what has the biggest impact in the game. Jason __________________ "I pretty much popped everything cold turkey. We were doing steroids they wouldn't give to horses." -- Tom House "I was very fortunate to have a pitching coach by the name of Tom House...Tom, I really miss those days that we spent in the weight room and out on the field working together." -- Nolan Ryan's HoF Induction Speech

05-22-2002, 01:16 PM	#8
Jason Moyer Hall Of Famer Join Date: Feb 2002 Location: Pittsburgh Posts: 5,003	</font><blockquote><font size="1" face="Verdana, Arial">quote:</font><hr /><font size="2" face="Verdana, Arial">Originally posted by Sony: <strong>I hope you turned injuries off</strong></font><hr /></blockquote><font size="2" face="Verdana, Arial">I'm using the following settings: replay mode (no player dev) injuries off trades off endurance off Basically I just want to create a completely even playing field, so I can tinker with a few ratings and see what sort of overall impact they have. Something interesting from the defensive study I'm doing now - all things being equal, a SS or 2B with A range is worth 10 more wins per season than a SS or 2B with E range. Considering that an entire team of GREAT clutch performers is only worth 10 wins over a team of Suffers clutch performers, that is a lot of bang for the buck. Jason __________________ "I pretty much popped everything cold turkey. We were doing steroids they wouldn't give to horses." -- Tom House "I was very fortunate to have a pitching coach by the name of Tom House...Tom, I really miss those days that we spent in the weight room and out on the field working together." -- Nolan Ryan's HoF Induction Speech

05-22-2002, 05:53 PM	#9
Carplos Hall Of Famer Join Date: Feb 2002 Location: Idaho Posts: 2,688 Infractions: 1/0 (0)	</font><blockquote><font size="1" face="Verdana, Arial">quote:</font><hr /><font size="2" face="Verdana, Arial">Originally posted by Jason Moyer: <strong>Also, FWIW, I'm running some more studies now to demonstrate: 1. The impact of A vs E defense at each position. 2. The impact of a single player who is twice as good as the league at all hitting ratings 3. The impact of a single player who is half as good as the league average at all hitting ratings My final report will be a guideline on building the ultimate compromise lineup, assuming that each player can only be good at one thing (either A/A fielding with half of average hitting or E/E fielding with twice the average hitting), with some studies on how each of my compromise ideas would actually perform on the field. I'll probably do some studies on the impact a twice-average pitcher in each role has, and the impact of speed. Then I can do another summary on what has the biggest impact in the game. Jason</strong></font><hr /></blockquote><font size="2" face="Verdana, Arial">That takes a lot of the fun out of the game..

05-22-2002, 09:52 AM	#1
Jason Moyer Hall Of Famer Join Date: Feb 2002 Location: Pittsburgh Posts: 5,003	Study #1 - What is clutch worth in OOTP4? I created a 2-team test league, both teams exactly the same, and ran a series of tests to determine the effects of clutch. Here are the results. In all of these tests, Team B is exactly like Team A with all players having their clutch rating set to "normal". The record after 10 sims is in parenthesis: Team A Normal - 81.5 Wins (5-4-1) Team A Great - 84.6 Wins (4-5-1) Team A Suffers - 75.2 Wins (1-9-0) The conclusion? Suffering in clutch situations has more of an impact than being great in clutch situations. Being great in the clutch, based on this study, is worth about 3-4 wins a year. However, notice the sub-.500 win/loss record. My conclusion is that there really is no big advantage to having a great clutch rating over a normal clutch rating. Suffering in the clutch is a different story. Based on this study, a team with all poor clutch performers would have 5-6 extra losses a year against normal competition. Even more telling is the win loss record of 1-9. My conclusion? Don't field teams of 25 guys who suffer in clutch situations. Unless, of course, they're all superstars. Overall, based on this study, I would say that clutch ratings have some slight effect, primarily negative, but in practical terms, when choosing whom to sign or put in your lineup, I would use it only as a tiebreaker between 2 players with exactly the same abilities. To conclude, clutch hitting may exist, but if it does its effects are negligible. Jason <small>[ 05-22-2002, 06:16 PM: Message edited by: Jason Moyer ]</small> __________________ "I pretty much popped everything cold turkey. We were doing steroids they wouldn't give to horses." -- Tom House "I was very fortunate to have a pitching coach by the name of Tom House...Tom, I really miss those days that we spent in the weight room and out on the field working together." -- Nolan Ryan's HoF Induction Speech

05-22-2002, 12:58 PM	#3
JAttractive All Star Starter Join Date: May 2002 Location: St. Catharines, Ontario Posts: 1,135	Is their a scale of Clutch-Suffer or is this a yes-no type thing? I have a couple players who are listed as normal but every playoffs they hit a ton of homeruns and their average is about .100 points higher. I was sure they were "nearly" clutch or something. I have noticed a huge difference in the playoffs though in my players listed as clutch vs. suffers, however I haven't played enough games to be accurate nor do I play many suffers-types so it is not really a fair comparison. Still I am seeing a trend in my limited games.

05-22-2002, 01:01 PM	#5
JAttractive All Star Starter Join Date: May 2002 Location: St. Catharines, Ontario Posts: 1,135	Oh and I notice the AI almost always brings in my Clutch reliever in the playoffs for important situations (I tend to let the AI handle pitching until late in the game), even though I have better pitchers on the bench that it went to all season. This guy is used nowhere near as much during the regular season as he is in the playoffs. Again this could be due to limited testings/games but it is possible their is a significant enough advantage to using them in playoffs.

05-22-2002, 01:01 PM	#6
Sony Minors (Single A) Join Date: Apr 2002 Location: Lafayette. LA Posts: 65	I hope you turned injuries off

05-22-2002, 06:59 PM	#10
Jason Moyer Hall Of Famer Join Date: Feb 2002 Location: Pittsburgh Posts: 5,003	Howso? From what I can tell, everything I've said is stating the obvious and applies to real life. 1. The defensive study just basically reaffirms the defensive weights Bill James uses, with the catcher being a notable exception, altho a good quality catcher being rare has more to do with the athletic requirements for the position rather than the importance of having a good defensive catcher. 2. A study I haven't published yet shows that A steal/A speed guys aren't really worth anything in terms of producing wins. Again, Bill James has been saying this for 20 years about real baseball, so no big surprises. 3. If I do a study comparing the impact of starters, relievers, and closers, and it shows that the starters have the most impact, followed by the middle relievers, followed by the closers, it would just show that the game is pretty close to reality. The stuff I'm been testing so far is pretty obvious imho. I'm actually surprised at how close the game mirrors real life in some respects. Jason __________________ "I pretty much popped everything cold turkey. We were doing steroids they wouldn't give to horses." -- Tom House "I was very fortunate to have a pitching coach by the name of Tom House...Tom, I really miss those days that we spent in the weight room and out on the field working together." -- Nolan Ryan's HoF Induction Speech

11-26-2003, 12:19 PM	#11
William4192 All Star Starter Join Date: Mar 2002 Location: Florida Posts: 1,625	Great Thread! __________________ What is this about Rhode Island? It is neither Road or Island. Come check out the PBA. http://www.pioneerbaseballassociation.com Or email at pbacommish@pioneerbaseballassociation.com

11-26-2003, 12:21 PM	#12
Gastric ReFlux Hall Of Famer Join Date: Oct 2002 Location: Member #3409 Posts: 8,350	This thread is clutch.

11-26-2003, 12:24 PM	#13
jaxmagicman Hall Of Famer Join Date: Jun 2003 Location: Retired defloration-maker living in Myrtle Beach, SC Posts: 7,801	Can we try this study with OOTP5? __________________ See ID Major League Baseball trademarks and copyrights are used with permission of MLB Advanced Media, L.P. Minor League Baseball trademarks and copyrights are used with the permission of Minor League Baseball. All rights reserved.

11-26-2003, 12:46 PM	#15
fat dude Major Leagues Join Date: May 2003 Posts: 392	Am I understanding you correctly when I read 10 games to mean you only simmed 10 games to come to your conclusions? I hardly think 10 games is enough to prove a thing. I think you're on the right track, but I would need to see a whole season simmed before I made any conlusions. Maybe I'm reading this wrong though.

11-26-2003, 01:55 PM	#18
ssmarsh Minors (Double A) Join Date: Jun 2003 Posts: 178	I took it to mean 10 games.

11-26-2003, 02:04 PM	#19
Henry Hall of Fame Join Date: Dec 2001 Posts: 6,498	It looks like 10 games given the win-loss records. I would think at least 2-3 seasons would be necessary.