I believe that it is largely a cognitive bias that keeps us from knowing the future. So before an effective methodology can be established for predicting the future, we have to eliminate or, at least minimize the belief that this simply cannot be done. So consider the following scenario. I have a large bowl of BBs in my hand and I say to you "I am going to turn this bowl upside down. Do you know what will happen ?" With great confidence you assert that you do know what will happen. The BBs will fall to the floor. There are three important observations to be made from this example.
First, your confidence in your knowledge is merely a matter of experience. The laws of gravity are validated hundreds of times each day so we believe them to be laws of nature that exist independent of our experience rather than hardened (meaning tested) experience. Hence, when you say "I know X" you are really saying "Based on past experience, I am confident that nothing will be revealed to me to suggest that X is not the case." Hence, knowledge is always speculation and the harder (more tested) the knowledge, the less likely it will be overturned. Thus, there is no difference between knowledge of the past, present or future as long as there is some method for increasing one's confidence that one's knowledge will not be overturned at some point with additional information.
For the second point assume that each BB has a number on it. I ask if you know what is going to happen if I turn the bowl over. You say "Of course I know !" I say "Great ! Where will BB #12,831 land ?" or if I really wanted to be difficult I might say "O.K. How much did the universe expand in the time it took the BBs to fall to the floor ?" To either question you would most likely react with great surprise thinking that I was being unfair or pulling something sneaky. The point is that there are always a wide variety of temporal, spatial, causal and contextual simplifying assumptions implied in the verb "to know". When we make claims about the present or past we are comfortable with the limitations of these assumptions. But when we talk about the future, any assumptions we make sound like hedging.
If I ask you "Do you know what happened in 1865 ?" You will say "Of course I do. The civil war ended" If I say, "No I mean what happened to interest rates." You will look at me like I am crazy. Somehow, I have been unfair again. Now assume I make the claim, "I know what will happen in the 21st century." And you ask the question "What will happen to interest rates in the second decade ?" If I cannot answer this question, you will think you have refuted my claim. Our cognitive bias leads to many such asymmetries in our respect for knowledge of the past versus knowledge of the future.
The third point is that you always assume that you have all the requisite information to make a reasonable claim. If the BBs were helium filled plastic balls or if there were a very strong electromagnet in the ceiling you would feel tricked. Thus, when you say "I know X" you are saying "I believe X to be the case, however, there may be an additional factor of which I am unaware that causes X not to be the case."
Thus, knowledge is a function of repeated experience. The meaning of the verb "to know" carries a variety of semantic assumptions. And knowledge is always contingent. When studying the past or the present we have a wide variety of techniques that we use to mitigate the effects of these vagaries of knowledge. We call these techniques research methods, critical thinking, or sometimes, philosophy of science. These techniques did not always exist. They were developed over time to increase the quality of our knowledge. All of these techniques eventually reduce to making assertions, challenging them, and seeing if they hold up. The techniques vary, of course, by situation. However, there is no reason why similar techniques cannot be applied to the future. There is a much greater variety in the quality of knowledge within a temporal frame than there is across temporal frames. Hence, the only reason we cannot "know" the future is that we really haven't tried.
