On 1 Sep 2003 23:50:28 GMT, Brian Sandle
wrote:

In sci.agriculture Torsten Brinch wrote:
On 1 Sep 2003 11:52:50 GMT, Brian Sandle
wrote:
No you need to be quite good at the subject to deal properly with
outliers. I am thinking that sometimes people do not eliminate them when
they should be, and others do but don't acknowledge it.

Imagine you represent a conservative govt applying as little as possible
health funding to a village of 100 people of mainly low income, based on
whether they can pay for it themselves or not. When calculating the
average will you include the income of the one multi-millionaire in the
village? That would make the average income rather higher, so you can fund
less. But the other 99 people would have no ability to pay, consequently.
The place would become a real eyesore.

But that's not an outlier problem, Brian. Indeed, there's not much of
a statistical problem in it :-)

Except in that part of statistics is deciding what measures to use and
what to measure.

I must admit I find it the bigger problem in imagining myself as a
conservative govt representative.

You have sampled the whole population,
you know the income of each and every individual in it, you know their
average income. The average is the average. It is just not a very good
descriptor for what the politicians want to describe.

Then you might change to the middle income. That might not work either if
there is a big tail of very low incomes.

Perhaps, could we use some statistics on misuse of the public health
care system? You know, some people go to the doctor for no good reason
again and again, etc, we could get some numbers on that. We could say:

This wouldn't be a problem in a user-financed system. And, "the money
lies best in the citizen's pocket". There must be a basic health
system, but need it cost that much? We could talk: about healthy
competition in the health care industry, people choosing freely
between products on a free market for a wealth of health services.
Everything would become cheaper then, our society would be richer,
and we could all get better health care than we get it now for less
money, or a faster car. Clearly -no one- benefits from an inefficient
tax-funded health system -- [this is where we show a colorful graph of
our statistic on-screen] -- a public health sector, which is so
vulnerable to overuse and misappropriation of resources.

Then if you were looking for how much the village could potentially donate
to a cause would the high earner still be an outlier?

Yes. In that situation average income might be a more suitable
descriptor. But again, this is not an outlier problem. The high earner
is known to be part of the population studied, so the data point
representing his income can never be considered an outlier.

So you might change from a purely latitude and longitude basis for the
sample to some other. Perhaps it is the subset of employees in the region.

Say you wanted to persuade people that potatoes in general are not high on
solanine. How many sweet potatoes are you allowed in the sample?
Given the figures the sweet potatoes might appear as outliers. This might
lead back to calling into question whether a sweet potato is a potato.

I think we -must- have decided on that question way before looking for
outliers in our data. :-)

I think Gordon has a little point, that he needed to be told a bit more
about the outlier categorizing.

But, what if all he wants to know is that Ms.Ingham is a witch
and should be burned?

But when you search the web for how
frequently `Monsanto' occurs in studies mentioning outliers, how much do
you get?

Interesting problem. How do you best restrict a web search, to make it
return only studies?

Heh. Over at sci.ag. Gordon has posted this Monsanto funded study,
which, :-) hold on to your chair, apparently has discarded outliers,
without mentioning that it has done it , far less telling when or why.

You should see Gordon, he is really -rough- now on that poor Monsanto
report now because of that :-)

Just kidding. Gordon has studiously not said a word about it.