Is it possible to prove randomness is not random?

[Kimi]

Is it possible to show that events that appear to be random are not actually random? Either mathematically, statistically, or even just with wordy explanation.
Maybe like applying chaos theory to random systems.
Or explaining that stock market that appears to be random actually follows the Markov process.

[Strong Eagle]

You can apply a number of statistical constructs to a data series to determine its degree of randomness. Your mobile phone works on this premise... to pull out the non random signal from random noise which is many many times more powerful. The mathematics can be quite hairy... the hardest course I took in college was a digital communications theory class.

As for the stock market, there are a million technical analysis pieces of software out there, all with the same idea that a discovered pattern can be extrapolated into the future. In many cases, the patterns are bogus. The real problem though is that the stock market might react completely differently tomorrow and your pattern would be shot to hell.

For a simple introduction to detecting separating signal from noise, you can look at my old website pages at

http://www.herberts.org/wayne/proj431/projmain.html

and

http://www.herberts.org/wayne/proj431/projres.htm

This was the introductory material to digital communications.

Now, this isn't purely random but it is a case where the noise is statistically uncorrelated with the signal... this filter will not work for a purely random noise signal.

[Strong Eagle]

Would you consider it random if you flipped a coin 100 times and you got a series or two where the coin came up heads 9, 10, or 15 times in a row?

A statistics prof I know made this assignment at the first of each semester. Flip a coin 100 times and record the results. He always knew who did it and who just made up the answers because the people who made up the answers couldn't imagine 10 heads as being random.

[ksl]

You can apply a number of statistical constructs to a data series to determine its degree of randomness. Your mobile phone works on this premise... to pull out the non random signal from random noise which is many many times more powerful. The mathematics can be quite hairy... the hardest course I took in college was a digital communications theory class.

As for the stock market, there are a million technical analysis pieces of software out there, all with the same idea that a discovered pattern can be extrapolated into the future. In many cases, the patterns are bogus. The real problem though is that the stock market might react completely differently tomorrow and your pattern would be shot to hell.

For a simple introduction to detecting separating signal from noise, you can look at my old website pages at

http://www.herberts.org/wayne/proj431/projmain.html

and

http://www.herberts.org/wayne/proj431/projres.htm

This was the introductory material to digital communications.

Now, this isn't purely random but it is a case where the noise is statistically uncorrelated with the signal... this filter will not work for a purely random noise signal.

As for the stock market, there are a million technical analysis pieces of software out there, all with the same idea that a discovered pattern can be extrapolated into the future. In many cases, the patterns are bogus. The real problem though is that the stock market might react completely differently tomorrow and your pattern would be shot to hell.

I find your opinion interesting, and not all wrong, T/A does have specific patterns, that can be monitored to minimise risk, not to predict. what will happen tomorrow.

But will predict, based on cyclic measures, return spikes, how T/A is used is also a million to one chance that two people plot the exact same, based on the evidence of other research data collected.

Sept the 11th was a prime example, it was unpredicatble, and the stocks crashed, people bailed out, I bailed in, based on the T/A the market will return to normality, but it's not only the main markets, it is the companies shares, one also must monitor.

what T/A is doing is minimising the risk, it is not a crystal ball to read the future, but a tool suggesting that if everything remains stable, the cyclic pattern will return to normality, eventually, and peaks and falls can be read, for the benefit of those, that know how to use them.

And to prove the point, you only have to plot, every stockmarket crash, to show that it does return.

The ones, that lose mostly are margin traders in futures, Normal shares are pretty safe, providing you know, which ones to buy, and at what price to but at.

Actual product knowledge and the ability to read the futuristic data, is an art, that also minimises risk.

Take the LCD TV panels of today, absolutely no match, against a high quality tried and tested CRT TV, The technology advancement is not due, to match CRT quality until mid 2006, then it is in it's project stage.

Now going back to sept 11th stock at rock bottom, on the T/A sheet. if you had purchased 300,000 shares at the price they was in September in LCD TV, 3009.tw 2409.tw 6116.tw and, you would be a pretty wealthy guy today.

The investment is sound all the companies are making profits, but margins are rock bottom. 7g technology is to expensive for the biggest company to invest in, So the talk is that all the companies will take a share in the 7g factory, and margins will be controlled.

The only thing T/A is telling us is the market will return to normal, providing there are no unforseen setbacks, like oil, wars, disasters, ect ect. In my opinion T/A is a great tool to minimise risk, and not a tool to read the future. It's always worked for me, in relation with other data, and I have been short term trading for 5 years, without any loss at all.

Main point, is the pattern is shot to hell, for a very short period in time, and is always a very good time to average down. when the market drops, you will never get a cheaper price. T/A proves the market will return. But it's not a predicting machine, and that's where many fail. it's a tool to minimise risk.

[ksl]

SE the link didn't work on the second truck sound, with filter!

[Strong Eagle]

ksl,

We are in agreement on this one. It is a tool to minimize risk. Where people go wrong is extrapolating a current trend into the future. Market trends are a follower not a predictor of economic activity. So when people do a Fourier analysis on a stock market data series, all they are seeing is what the market would look like IF all the variables remain the same.

My sense was that Kimi was looking at the situation from the perspective of all the technical investing systems being touted... you ever pick up one of those magazines? You'd swear all you need to do is follow a trend line and get rich.

Glad to know you are doing well in the investing game. I should buy you a beer and listen to your secrets.

PS: Don't know what happened to the sound files... the website has made a couple of moves from Rice since it was made.

[YF]

I will ask the magic 8-ball....No.

It may be similar to solving the so-called halting problem which you would need an 'oracle' to do it...I am not sure exactly how these two things fit together exactly, this is just my intuition talking.

RDL

[Guest]

After 100 tosses a coin should come up heads approx. half the time. The more tosses the closer you will get to exactly half. But there will be long strings of heads and tails during the process. That does not mean it is not random. To not be random you'd have to show that you can predict the outcome with better than 50% accuracy. This can be done if you figure a way to flip the coin exactly the same way every time. Otherwise, every toss has a 50% chance of being heads or tails and the coin has no memory of what happened previously. If the coin came up heads 10 times in a row your odds of correctly predicting the next toss are still 50%.

[Guest]

I'm sorry. You're coin theory is flawed. A coin, any coin will have symmetry imperfections, meaning it'll be baised either to fall heads that little bit more often or tails more often. It's a tiny bias but it will have significant consequence at 1.000.000 throws, i.e. no absolute randomness.

Second, the act ofl flipping a coin will also adhere to a bell curve of throws, i.e. a lot at very similar height, tapering off to higher and lower. The bell curve peak gives rise to more bias in randomness.

Since we're nitpicking here.

[Guest]

In Chinese mathematics everything is possible, that's why so many Asians addicted to gambling

[mrdodge]

ksl,

So when people do a Fourier analysis on a stock market data series, all they are seeing is what the market would look like IF all the variables remain the same.

.

Its absolutely correct to say that its impossible to fully characterise the behaviour of a stock market through simple models. The idea that many technical analysis schemes have that the market (ie stock index) somehow captures all the information that everyone used to make their decisions and there must be a way of unlocking that is fundamentally flawed, from a general viewpoint

Having said that, there are quite a few useful techniques that can give helpful information when you are trading. The most straightforward is GARCH - Generalized Autoregressive Conditional Heteroskedasticity - probably one of the most well understood stochastic models around. Think of it as a very sophisticated moving average that can give you signals about likely short term movements of stock prices. Google for it if you are interested.

You need a lot of firepower to use these sort of models - not only from the computational and mathematical sense - but using these sort of models for trading is really an arbitrage business if you a doing it long term, so you need a bunch of capital to take real advantage.

Did this sort of thing for while when working for a derivatives trading team in an investment bank. Lots of fun.

[ksl]

ksl,

So when people do a Fourier analysis on a stock market data series, all they are seeing is what the market would look like IF all the variables remain the same.

.

Its absolutely correct to say that its impossible to fully characterise the behaviour of a stock market through simple models. The idea that many technical analysis schemes have that the market (ie stock index) somehow captures all the information that everyone used to make their decisions and there must be a way of unlocking that is fundamentally flawed, from a general viewpoint

Having said that, there are quite a few useful techniques that can give helpful information when you are trading. The most straightforward is GARCH - Generalized Autoregressive Conditional Heteroskedasticity - probably one of the most well understood stochastic models around. Think of it as a very sophisticated moving average that can give you signals about likely short term movements of stock prices. Google for it if you are interested.

You need a lot of firepower to use these sort of models - not only from the computational and mathematical sense - but using these sort of models for trading is really an arbitrage business if you a doing it long term, so you need a bunch of capital to take real advantage.

Did this sort of thing for while when working for a derivatives trading team in an investment bank. Lots of fun.

Yes, I believe this to be an interesting subject for all, that dabble in the stock market, even though there are very many, that don't believe in T/A.

Patterns can be identified over a very short period, depending on ones motives for that particular time frame, combined with additional macro information, it becomes quite a powerful tool, to minimise risk.

The only problem I have found, is placing total trust in the concept, when dealing large value, the gut feeling tends to take over. In the early days, I made quite a few errors, which I quickly resolved.

1. Never be to greedy! it is normal to see a bull run for 2 to 3 days, before dropping. I can only refer to TSE, which is limited 7% movement.

So I also stick to a 10 to 15% gain, sell and wait for the drop, before coming in again on the same share, Techs are cyclic, so T/A is a great tool, for the seasonal swings. Maybe expatsforums, should consider a thread on investment, I am sure it would increase membership, and is great fun to discuss stocks, and ideas. Sorry for drifting off topic.

[mrdodge]

One precept that I fully subscribe to is run your winners and cut your losses.

The best traders I know are absolutely ruthless about cutting losing trades. Take it on the chin and move on - staying attached to a losing position only means you'll lose more. While there are always exceptions you can't run a business on them so you have to play the odds.

The other real trick, of course, is to know when to cash out. Running your winner too far invariably turns it into a loser.

Its a combination of experience, courage, market knowledge - traders instinct, I guess. Some have it (not me!) and some don't

[ksl]

One precept that I fully subscribe to is run your winners and cut your losses.

The best traders I know are absolutely ruthless about cutting losing trades. Take it on the chin and move on - staying attached to a losing position only means you'll lose more. While there are always exceptions you can't run a business on them so you have to play the odds.

The other real trick, of course, is to know when to cash out. Running your winner too far invariably turns it into a loser.

Its a combination of experience, courage, market knowledge - traders instinct, I guess. Some have it (not me!) and some don't

Without a doubt! I learn't my lessons the hard way. My wife bare witness to the fact, that I was down a 30 thousand Sing$ dollars, when I cut lose, she thought I was silly, But I'd already pulled data, to cut lose, and I did.

I must say it's not easy when it's a lot of money, but I'd already identified, a back up. I normally track monitor 30 to 40 in my sector, and manage 10 normally.