[home][rumors and news][model release matrix][dealer network][desktop calendar][exhaust notes][tov forums][links][search][sponsors][garage][login]

Tire Rack Upgrade Garage
 Search for a Dealer:
 Canadian Flag US Flag
 Honda Acura
 ZIP  
American Honda Sets New June Sales Records on Strength of Light Trucks
More.......................
Honda reveals refreshed 2019 HR-V and Pilot
More.......................
Acura TLX GT wins class and places 3rd overall at 2018 Pikes Peak International Hill Climb
More.......................
Spied on the Street! 2019 Pilot and Pilot PHEV mule
More.......................
All-New 2019 Honda Insight Brings Style, Sophistication and 55 mpg City Rating
More.......................
It's Official: Honda to supply Red Bull Racing in Formula One from 2019
More.......................
Will the next-gen TLX bring back double wishbones? Or a Plugin Hybrid model?
More.......................
Acura tackles Pikes Peak with 2019 RDX, NSX, and TLX entries
More.......................
Civic --> Re: 2017 Civic Si 360whp One Take
Join Discussion......
Type R --> Re: Such a damn nice car
Join Discussion......
RDX --> Re: New RDX vs NX Infomercial
Join Discussion......
General Talk --> Re: 2018 WC
Join Discussion......
ILX --> Re: 2019 ILX
Join Discussion......
Professional Motorsports --> Re: MotoGP - Sachsenring TT (you should watch)
Join Discussion......
General Talk --> Re: Trying to decide on color
Join Discussion......
Fuel Cell Technology --> New H70 pumps freezing
Join Discussion......
Amateur Racing & Driving --> Re: Car recommendation for cheap track fun?
Join Discussion......
Accord --> Re: Just got a 2.0 Accord Touring
Join Discussion......
Accord --> Re: Acclaimed 2018 Accords sit on the lot
Join Discussion......
General Talk --> Celaya Plant Flooding and Recovery
Join Discussion......
IMA/Hybrids --> '18 HAH vs '17 HAH
Join Discussion......
Fuel Cell Technology --> Re: Marauding Mirais
Join Discussion......
General Talk --> Re: ROAD TRIP... SOB Oregon cops
Join Discussion......
First Drive: 2019 Honda Insight
Read Article....................
2019 Honda Insight PR Photo Gallery
Read Article....................
First Drive: 2019 Acura RDX
Read Article....................
2019 Acura RDX Features & Specifications
Read Article....................
PR Photo Gallery - 2019 Acura RDX Advance
Read Article....................
PR Photo Gallery - 2019 Acura RDX A-Spec
Read Article....................

[fancy] [flat] [simple]
  TOV News > Virtual Dyno Comparo: 2011 Honda Civic Si vs 2012 Honda Civic Si > > Re: Eh, dyno isn't a reliable metric for comparison

Viewing Threshold (What is this?)

Thread Page - [1] 2
Author
    
sugaki
Profile for sugaki
Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 00:46
Reply to This Message Attach Quote to Reply
Sorry but Jeff from TOV doesn't seem to understand that even the same model of a car can give varying power output due to the engines being mass produced.

The more likely explanation is that the Civic Si is a "factory freak" rather than Honda covertly spending money to bump up the power while not advertising. No two cars will give the same exact dyno reading.

The purpose of a dyno is to test and analyze the trends with your own car. You can't come to any broadbrush conclusions comparing two separate cars on the same dyno.

danielavg1991
Profile for danielavg1991
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 00:50
Reply to This Message Attach Quote to Reply
But he said in the article that Shawn also dyno'd another 2011 civic somewhere else that put down more power than earlier examples. You don't think a 14 horsepower difference is pretty noteworthy?
montechester
Profile for montechester
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:11
Reply to This Message Attach Quote to Reply
danielavg1991 wrote:
But he said in the article that Shawn also dyno'd another 2011 civic somewhere else that put down more power than earlier examples. You don't think a 14 horsepower difference is pretty noteworthy?


Given the same Dyno, same operator setup, same callibration, similar weather conditions... 14 HP is absolutely significant. BUT, if any or some of these factors are not the same, then it's pretty hard to compare. I'm sure Jeff did everything he could to make it an Apples to Apples comparison.

I'm just still hoping the new version of the 2.4 in the '12 Civic will spin a bit harder than people are assuming at this point. My sources can't tell me yet what the redline is going to be.

CarPhreakD
Profile for CarPhreakD
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:19
Reply to This Message Attach Quote to Reply
The dyno is the "only" reliable metric to compare engines. If the engines are produced with any semblance of quality they would have similar outputs regardless.

However, because this is only a virtual comparison, you cannot put much weight under the findings until both cars are tested on the same day.

notyper
Profile for notyper
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:21
Reply to This Message Attach Quote to Reply
I've dyno'd about 10,000 cars over the last 9 years. About half of them have been Hondas. I can tell you that for any given model, 95% of the examples tested tend to dyno within about +/- 1% of the average for that model.

On the 8th gen Si in particular, I've dyno'd a couple hundred (at least), and they're all within a couple hp of the average. Given that Jeff and I are on opposite coasts of the country, and dyno'd two different cars (one press, one independently owned), on two different dynos and saw double digit hp gains on the 2011 vs. the older cars, I think it at least bears investigation. Could they be a couple of flukes? Nothing is impossible. But it is extremely unlikely.

This kind of reminds me of when the 2004 S2000 was released with the F22C engine rated at the same hp as the older F20C. But we consistently saw the F22C putting down 10-15 hp more on the dyno than the F20C. A lot of people called BS, but it's a pretty well known fact now that the F22C is a lot stronger.

SC

JeffX
Profile for JeffX
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:24
Reply to This Message Attach Quote to Reply
montechester wrote:
danielavg1991 wrote:
But he said in the article that Shawn also dyno'd another 2011 civic somewhere else that put down more power than earlier examples. You don't think a 14 horsepower difference is pretty noteworthy?


Given the same Dyno, same operator setup, same callibration, similar weather conditions... 14 HP is absolutely significant. BUT, if any or some of these factors are not the same, then it's pretty hard to compare. I'm sure Jeff did everything he could to make it an Apples to Apples comparison.

I'm just still hoping the new version of the 2.4 in the '12 Civic will spin a bit harder than people are assuming at this point. My sources can't tell me yet what the redline is going to be.



Just to clarify, in the graphic I did not compare the '11 Si to any other Si in similar conditions. What the chart shows is the '11 Si vs the '11 TSX, with the dyno events separated by two weeks. The weather was slightly cooler on the day I dyno'd the Si, but that means there was a negative correction factor for the Si. I was simply commenting that the reason I got the Si loan from Honda was to check to see if what Shawn had observed was a fluke or not. Compared to the other results I've seen on Dynolab's dynojet, the '11 Si that I tested was considerably stronger. All the other pre-11 Sis that I have seen tested there have been in the very low 180whp range. While far from conclusive, it's notable that of a sample of TWO 2011 Si's that TOV has tested, both were notably stronger than earlier examples we have tested. At his shop, Shawn has probably tested Si's numbering in the dozens or even hundreds, so when he told me about the surprising results of the one stock 2011 Si that he tested, I decided to check on my own here. Otherwise, I never would have requested a media loaner of the '11 since there were no announced changes.

I should note that the results on that chart for the Si were actually corrected DOWNWARDS, with a correction factor of 0.98 due to the temperature conditions during the time of dyno testing. The TSX was a 1.00 CF because it was slightly warmer. The uncorrected figure for the Si was actually a peak of over 195whp. Remember, the 2011 Si is rated at 197hp at the crank.



Last edited by JeffX on 02-18-2011 01:29
danielavg1991
Profile for danielavg1991
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:37
Reply to This Message Attach Quote to Reply
Is it possible that Honda did this to offset the marginal weight increase since 2006? I believe the car has put on a few pounds in the later model years. I'm also assuming the car doesn't feel any quicker on the road...
JeffX
Profile for JeffX
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 01:48
Reply to This Message Attach Quote to Reply
danielavg1991 wrote:
Is it possible that Honda did this to offset the marginal weight increase since 2006? I believe the car has put on a few pounds in the later model years. I'm also assuming the car doesn't feel any quicker on the road...


You're right - the '11 Si I tested didn't feel any quicker on the street than any other previous Si I had tested, so I was a little bit surprised when I saw the dyno results. Now I wish I had bothered to weigh it to see if it was considerably heavier than earlier models.

sugaki
Profile for sugaki
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 02:10
Reply to This Message Attach Quote to Reply
notyper wrote:
I've dyno'd about 10,000 cars over the last 9 years. About half of them have been Hondas. I can tell you that for any given model, 95% of the examples tested tend to dyno within about +/- 1% of the average for that model.


And 5% is enough to say that it's possible. At the very least, don't think it's at all reliable to make a pronouncement that 11 Si's have better powertrains than 06-10 simply cus a couple dynoed higher. You'd need a lot stronger correlation than that.

With the S2k, you're talking about two different engines with different displacement. Unless you were to dismantle the K20Z3 and establish that the internals are somehow tweaked for more power, it's too premature to jump to any conclusions.

And to another post, no, the purpose of the dyno isn't to compare two different engines, especially two different cars. You have loss from friction, difference of amount of oil in the crank case, etc. The point of dynoing is to track changes with your own car for sake of tuning.

notyper
Profile for notyper
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 02:26
Reply to This Message Attach Quote to Reply
Are you familiar with the methodology behind determining standard deviations and what that represents?

While we haven't seen a big enough sample set to be sure yet (at least on 2011's), we're talking about a difference of 4-6% on the two cars we've tested. If 95% fall within +/- 1%, that means that the standard deviation of the distribution is 0.5%, or about 1 hp. That means that over 99% should fall within +/- 3hp. To be 10-12 hp higher is well outside the bounds of normality. The odds of having two such cars in a sample set of hundreds of Civics is very low. The odds of those two statistical outliers both being 2011 models out of the entire population of 2006-2011 Civics is even lower.

Again, I'm not saying they aren't flukes. Just saying that it is unlikely.

SC

bluefz22
Profile for bluefz22
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 03:06
Reply to This Message Attach Quote to Reply
Naw,man. This is a test of an 11 SI vs. an 11 TSX. I'm waiting for a real 2012 SI test.
Grace141
Profile for Grace141
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 08:28
Reply to This Message Attach Quote to Reply
CarPhreakD wrote:
The dyno is the "only" reliable metric to compare engines. If the engines are produced with any semblance of quality they would have similar outputs regardless.

However, because this is only a virtual comparison, you cannot put much weight under the findings until both cars are tested on the same day.



With the given set of parameters I too would say the results are significant. The '11 Civic was supposed to be the model change year. Does anyone know of new part numbers for components in the '11 Si transmission, FI system, or computer? Is it feasible they may have early-adopted in the '11 Si some of the low friction tech they are saying resulted in MPG gains in the TSX and Accord? The increased HP you folks are talking about are wheel HP gains and not crankshaft gains, right?

typer_801
Profile for typer_801
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 09:52
Reply to This Message Attach Quote to Reply
Looks like it's time to start scouring parts diagrams to see if there are any differences in part #'s for any key items (cams, t-body, intake, head casting, pistons, intake, etc.) which may explain the gains of the 11 Si's vs. prior years.


JeffX
Profile for JeffX
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 12:23
Reply to This Message Attach Quote to Reply
bluefz22 wrote:
Naw,man. This is a test of an 11 SI vs. an 11 TSX. I'm waiting for a real 2012 SI test.


because the results are going to be WAY different.

JeffX
Profile for JeffX
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 12:35
Reply to This Message Attach Quote to Reply
typer_801 wrote:
Looks like it's time to start scouring parts diagrams to see if there are any differences in part #'s for any key items (cams, t-body, intake, head casting, pistons, intake, etc.) which may explain the gains of the 11 Si's vs. prior years.




I checked the cams, t-body, and intake manifold and they're all the same part numbers (at least according to the online parts store I referenced). So I guess that leaves the intake, exhaust system and catalytic converter. Of course, the engine flash too. I know a flash in a normal Civic Si will help pick up a little bit of peak HP (not 12whp) and a nice gain in midrange and peak torque, so maybe Honda worked to optimize that PLUS opened up the intake and/or exhaust a little bit.

sugaki
Profile for sugaki
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 15:44
Reply to This Message Attach Quote to Reply
notyper wrote:
Are you familiar with the methodology behind determining standard deviations and what that represents?

While we haven't seen a big enough sample set to be sure yet (at least on 2011's), we're talking about a difference of 4-6% on the two cars we've tested. If 95% fall within +/- 1%, that means that the standard deviation of the distribution is 0.5%, or about 1 hp. That means that over 99% should fall within +/- 3hp. To be 10-12 hp higher is well outside the bounds of normality. The odds of having two such cars in a sample set of hundreds of Civics is very low. The odds of those two statistical outliers both being 2011 models out of the entire population of 2006-2011 Civics is even lower.

SC



Yes I'm familiar with statistical analysis--and it's why your conclusions are off-base and not grounded at all.

A difference of 4-6% on an utterly minuscule sample sizing doesn't say much. To say 95% fall within +/- 1% is sheer conjecture. If you're wanting 95% confidence within +/- 1%, you'd need to sample a lot more cars, easily over half. 2 cars will NOT give a normal distribution at all.

Let's assume Honda made 1000 Si's in 2011 (have no idea what the actual number is). When you sample 2 cars with 95% confidence, you're going to have a margin of error (confidence level) of 69.26%. From a statistical perspective, those 2 dynos mean nothing. Since the 2011 Si is purportedly making 10% more power, you'd need to sample enough cars to have the margin of error be lower than 10%.

To have a margin of error of 5% (which is the standard), you'd need to sample 278 cars, which is why I don't see how you can say 95% fall between +/- 1%.

Bringing statistics into the equation only hurts your point.

notyper
Profile for notyper
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 16:03
Reply to This Message Attach Quote to Reply
I'm talking about a sample size of 300+ Si's since 2006. Since Honda has announced no engine changes, nor ratings changes in that time, that's a pretty reasonable sample size. The only way you can't group all of them together is if the 2011's are different and Honda made an engine change, in which case the additional power is perfectly understandable and your concerns are moot.

Using a sample set of 300 (I'd have to go count, but I believe it is higher - I've dyno'd 5 Si's in the last few days alone) and a population of US market Si's made since 2006 of 100,000 units and 99% of sampled units falling within +/-3 hp, we get a confidence interval of +/- 1.5. A very good confidence interval by any measure.

Clearly the 2011 data is well outside any reasonable statistical limits, which implies a change in engine configuration. I have people checking with Honda Manufacturing and I'm sure Jeff has extended feelers as well.

SC


sugaki
Profile for sugaki
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 17:59
Reply to This Message Attach Quote to Reply
notyper wrote:
I'm talking about a sample size of 300+ Si's since 2006. Since Honda has announced no engine changes, nor ratings changes in that time, that's a pretty reasonable sample size. The only way you can't group all of them together is if the 2011's are different and Honda made an engine change, in which case the additional power is perfectly understandable and your concerns are moot.


You're still not using statistical analysis correctly. The main argument you're making here is that the 2011 Civic Si makes 14 more HP. The focus is on the 2011 Civic, not 2006-2010. The fact that 06-10 Civic Si's make 180ish whp isn't being debated--that's not the hypothesis. Hence sample sizing of the 2006-2010 Civic is irrelevant since you're not trying to prove anything with those years.

You're making a hypothesis about the 2011 Civic Si, therefore the sampling has to come from the 2011 year.

owequitit
Profile for owequitit
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 18:17
Reply to This Message Attach Quote to Reply
sugaki wrote:
notyper wrote:
I'm talking about a sample size of 300+ Si's since 2006. Since Honda has announced no engine changes, nor ratings changes in that time, that's a pretty reasonable sample size. The only way you can't group all of them together is if the 2011's are different and Honda made an engine change, in which case the additional power is perfectly understandable and your concerns are moot.


You're still not using statistical analysis correctly. The main argument you're making here is that the 2011 Civic Si makes 14 more HP. The focus is on the 2011 Civic, not 2006-2010. The fact that 06-10 Civic Si's make 180ish whp isn't being debated--that's not the hypothesis. Hence sample sizing of the 2006-2010 Civic is irrelevant since you're not trying to prove anything with those years.

You're making a hypothesis about the 2011 Civic Si, therefore the sampling has to come from the 2011 year.



Actually, the entire premise is 2011 vs 2006-2010. So far, based on the widely known numbers of the 2006-2010 Si, it is quite clear that both 2011 SI's tested were significantly outside the norm. Granted, 2 is not a huge or conclusive sample, but it is indicative of a potential trend. The fact that 2 randomly sampled cars both dyno'd similarly and were BOTH outside of the norm for the 06-2010 could very well indicate that it is in fact, stronger.

The main that you are neglecting is that Shawn and Jeff never said it was 100% sure, they merely said it was odd that both cars sampled higher. They then said that it warranted further inquiry, which it does. There is nothing flawed about the analysis, because his numbers on the 06-10 are well known, and he has made no conclusions about the 2011. They have merely discovered an apparent difference, and have said it should be looked into. They have accounted for potential errors, and are using sound methods. Also, Shawn clearly said that it is not impossible that both cars were ringers, but it is unlikely.

Your premise about the hypothesis is flawed because the hypothesis was relative to 06-10 models, which means they MUST be included in any such comparison.

Grace141
Profile for Grace141
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 18:19
Reply to This Message Attach Quote to Reply
So sugaki is saying a sample of 2 cars out of all 2011 Si's is insignificant and notyper is saying that 300 cars out of 302 is significant. Good times. Either way, assuming all else is equal the increase in power without admission from Honda is a big deal. I wonder if they knew about it?

So we're talking about:

- Similar fuel used in all cars.
- Are the dynos used of the rolling road type or the bolt-to-the-hub type?
- No evidence of management software updates.
- No noticeable changes in components.
- No changes in testing procedures such as throttle settings.

I'm thinking that once the mystery is solved there is a large group of Si owners out there that might be able to get a nice bump in HP.




sugaki
Profile for sugaki
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 18:48
Reply to This Message Attach Quote to Reply
owequitit wrote:
Your premise about the hypothesis is flawed because the hypothesis was relative to 06-10 models, which means they MUST be included in any such comparison.


Sigh. The hypothesis is comparing to 06-10, but the data of the 06-10 is not in question, data for the 2011 is. Doesn't matter whether you sample 300 or 30,000 Civic Si's from 2006-2010, nobody is arguing that they make 180-ish whp. Hence 06-10 data is a red herring, and for the purposes of the hypothesis, irrelevant.

Yes, that 2 cars have posted higher numbers is intriguing, and it warrants investigation. It does show a correlation. But it's far from conclusive to the point of declaring that all 2011s are quicker, and that we ought to buy 2011s instead of 2006-2010.

vandme10
Profile for vandme10
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 18:48
Reply to This Message Attach Quote to Reply
sugaki wrote:
And 5% is enough to say that it's possible. At the very least, don't think it's at all reliable to make a pronouncement that 11 Si's have better powertrains than 06-10 simply cus a couple dynoed higher. You'd need a lot stronger correlation than that.



For arguments sake, here's what is actually written, versus what you're suggesting was written:

Jeff wrote:
The "bone stock" curve on the '11 Si I tested obliterates the results of my own 2008 Civic Si, so it seems like Honda has possibly made some unannounced enhancements to the K20Z at some point.


The keywords here are "bone stock" (specifically quoted by Jeff), "seems", and "possibly". At no point was it said that Honda unequivocally made changes. Jeff clearly stated he's checked most of the part numbers for the "externally visible" parts. All of them are the same, except the few he mentioned he did not check. He also mentioned it was possible those he did not check do have changes. But we can also surmise those he did not check could possibly NOT have changed.

Nobody's saying you're wrong, but it is apparent you've misinterpreted a minor tangent of a "fantasy dyno" (since TOV has not driven nor dyno'd an actual 2012 Civic). The point here is to temporarily satisfy curiosity about what the changes to the vehicle between a generational update could mean.

99Type_SH
Profile for 99Type_SH
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 20:21
Reply to This Message Attach Quote to Reply
Okay, had to chime in sugaki you have no idea what you are talking about. Your understanding of Credibility theory is as bad as your use of madeupstatistics. But your erroneous use of the word correlation was down right insulting. Please no more math talk out of you. If you were one of my students I'd suggest that despite your forum confidence you need special tutoring!

The odds that you independently draw two consecutive 4+ standard deviations moves from a sample mean are incredibly small. This leads anyone with common sense to realize they are likely no longer drawing from the same population. In fact if they had test data for just one more car with the same result you can pretty much book it that they are no longer drawing from the same population.

Provided the test statistics provided here from the dyno guys is correct, then with only two draws the safe money should be put on the assumption that the 11SI does indeed have more power than the 10SI.

notyper
Profile for notyper
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-18-2011 22:02
Reply to This Message Attach Quote to Reply
I don't know, it has been 10-12 years since I last taught statistics, but I didn't think they'd changed the basics since then.

I think you're twisted up about the premise, which is understandable. Others have made a go of explaining it, but let's try once more.

Honda has made no claims of performance improvements on the Civic Si engine (K20Z3) since it's introduction in 2006. This applies to all models inclusive up to the 2011 models currently on the lots. Thus, our population should include all K20Z3 Civics made up to this point.

Over the course of that model run, cars dyno'd have been remarkably similar. To make the comparison as useful as possible, I will only use my sample set since it was obtained on only my test equipment, eliminating many other variables. Previously in this thread I have outlined the details of this sample size, the mean, the standard deviation, etc.

The 2011 cars that, given Honda's complete lack of announcement of greater power, should dyno the same, so far have not. As part of the sample set of 8th gen Civic Si's, this is a problem. They are so far outside the statistical bounds of the population as to raise questions about, as another poster put it, whether they are part of the same population at all. It is highly unlikely that one, let alone two of these cars would both dyno significantly higher than the existing population if there have been no running changes in the drivetrain.

Thus, the hypothesis to be tested is that the 2011 cars have experienced some change in their build or specification because they do not appear to fit in the same population as the 2006-2010 cars. We will find out soon enough if that is true or not.

SC

sugaki wrote:

You're still not using statistical analysis correctly. The main argument you're making here is that the 2011 Civic Si makes 14 more HP. The focus is on the 2011 Civic, not 2006-2010. The fact that 06-10 Civic Si's make 180ish whp isn't being debated--that's not the hypothesis. Hence sample sizing of the 2006-2010 Civic is irrelevant since you're not trying to prove anything with those years.

You're making a hypothesis about the 2011 Civic Si, therefore the sampling has to come from the 2011 year.



Foof
Profile for Foof
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-19-2011 16:51
Reply to This Message Attach Quote to Reply
I am currently teaching stats at 2 levels. Graduate level over the summer.
Basic assumptions I’ve made are:

1. 181 is the true mean HP of a 2006-2010 Si.
2. The variance of both the dyno error measurement and the car’s error are finite. (This isn’t much of an assumption. If this wasn’t true, every 300th or so car that got dyno’d would have 10 times more power than the manufacture’s claim.)
3. The population of a car model has a standard deviation of around 2 HP. This might be off, as I am a stats dude, not a dyno dude.

So 192-181= 11 HP or, 5.5 standard deviations over the mean.

Then we can conclude, via Chebyshev's inequality, that the probability of an occurrence of 192 or higher is, regardless of the distribution of the errors, 1/(5.5^2) = 3.3% of the time.

Chebyshev’s inequality is VERY conservative.

A more reasonable assumption is that the errors are distributed approximately Gaussian (as the errors are most likely sums of very small oddities). . . in which case, the probability of seeing a car dyno at 189 (notice the number change) or higher when the base population has a mean of 181 is about 0.0032%. I’m not going to dig out my extended normal table to find an “act-of-god” probability associated with a z-score of 5.5.

Essentially, a sample of size 1 tells us that there is almost certainly something different, or the dyno's busted. Having two dyno's out in this region means we oughta find out exactly what is different.

sugaki
Profile for sugaki
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-19-2011 21:57
Reply to This Message Attach Quote to Reply
Foof wrote:
3. The population of a car model has a standard deviation of around 2 HP. This might be off, as I am a stats dude, not a dyno dude.

So 192-181= 11 HP or, 5.5 standard deviations over the mean.



Okay and this is where I firmly disagree. An earlier poster said 4+ standard deviations. Huh? Totally pulled that out of thin air. And since people are under the impression that every single car gives the same result on the same dyno given same mods, here's some actual dynos, from various Civic Si's--but all on the same dyno. Note this is done at 3000 ft, so the results are lower than what you'd get at sea level. Stock Si's make 180-ish, maybe even less.

Bone stock 2009 Civic Si making 191 whp (guess 09's a special year too?)


Civic Si tuned with Flashpro, exhaust and SRI making 184 whp. Shouldn't it be making more than the stock Si? That's 2+ standard deviations from the norm!


Yet another Civic Si with tuned Flashpro, SRI and exhaust. Making 194 whp.


Civic Si with just a CAI, making 186hp:


Which goes back to the point I made earlier. No car is the same, different ones respond differently to dynos. You have factory freaks, although typically they make 180hp.

99Type_SH
Profile for 99Type_SH
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-19-2011 23:30
Reply to This Message Attach Quote to Reply

LudegarH22A7
Profile for LudegarH22A7
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-20-2011 02:49
Reply to This Message Attach Quote to Reply
Bad tuna, eww.
Honda-D
Profile for Honda-D
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-20-2011 13:13
Reply to This Message Attach Quote to Reply
Dude, don't take it so serious.

Who cares. It's just a educated guess based on PR info. I really don't think Honda will do anything new to get more of this setup than the TSX. They don't have the time or money to spend on redesigning for the Civic, hence just pulling the 2.4L from the shelf.

So this dyno is a good indication of what's to come on the 2012 Si.

Of course, if Honda did end up still have a few trick on its sleeves to do better than that dyno, everyone can't be happier. We need sites like VTEC to keep calling them out.

owequitit
Profile for owequitit
Re: Eh, dyno isn't a reliable metric for comparison [View News Item]    (Score: 1, Normal) 02-20-2011 15:50
Reply to This Message Attach Quote to Reply
sugaki wrote:
Foof wrote:
3. The population of a car model has a standard deviation of around 2 HP. This might be off, as I am a stats dude, not a dyno dude.

So 192-181= 11 HP or, 5.5 standard deviations over the mean.



Okay and this is where I firmly disagree. An earlier poster said 4+ standard deviations. Huh? Totally pulled that out of thin air. And since people are under the impression that every single car gives the same result on the same dyno given same mods, here's some actual dynos, from various Civic Si's--but all on the same dyno. Note this is done at 3000 ft, so the results are lower than what you'd get at sea level. Stock Si's make 180-ish, maybe even less.

Bone stock 2009 Civic Si making 191 whp (guess 09's a special year too?)


Civic Si tuned with Flashpro, exhaust and SRI making 184 whp. Shouldn't it be making more than the stock Si? That's 2+ standard deviations from the norm!


Yet another Civic Si with tuned Flashpro, SRI and exhaust. Making 194 whp.


Civic Si with just a CAI, making 186hp:


Which goes back to the point I made earlier. No car is the same, different ones respond differently to dynos. You have factory freaks, although typically they make 180hp.



Nobody has questioned whether dynos vary from unit to unit. This is just silly.

What IS being questioned is the 11's relative performance compared to a huge sample of 06-10's, on the same dyno, over a large number of runs. They varied too, but relevant and valid averages were concluded over the course of more than 300 Civic Si runs. The fact that 2 of the tested Si's ran significantly higher than 1-2 standard deviations from the norm is not going to be washed away by posting random dyno plots. The fact is that in 2 cases, on 2 seperate dynos, both of which are well known quantities on this site, BOTH 2011 Si's, tested under similar conditions, settings etc were both significantly outside the statistical norm.

Now, if you can provide a similar sample set with included 2011 data that refutes this, on the same dyno, with the sample also coming from the same dyno, then you might be in business. Until then, it doesn't work.

It is not impossible that the test cars were anomolies. It is statistically unlikely. There is a difference.


 
Thread Page - [1] 2
Contact TOV | Submit Your Article | Submit Your Link | Advertise | TOV Shop | Events | Our Sponsors | TOV Archives
Copyright © 2018 Velocitech Inc. All information contained herein remains the property of Velocitech Inc.
The Temple of VTEC is not affiliated with American Honda Motor Co., Inc. TOV Policies and Guidelines - Credits - Privacy Policy
29 mobile: 0