Why Data Transition to Information to Knowledge can Fail Us?
Our world is drowning in data. We are collecting all kinds of data. Data we want to share and some we do not. And this is done almost automatically.
30–50 years ago data was so limited and rarely shared. We are living in the other extreme of that data swing.
Most people mistakenly believe that data is information. NO..NO..NO!!! Data can be turned into information after processing it. So do not get confused. YOu are growing in data and not information.
BUT information is just the start towards understanding. It means that you understand the inputs and system enough to be able to predict the outcomes. The best way I can explain it is with an equation.
Suppose we have x=3, is one point of our data, and our information is the equation y=x²+5. Then we can solve for y as y=(3)²+5=9+5=14.
At this point, we have used our data to possibly predict the outcome y based on the equation, information.
BUT how do we move from information to knowledge?
For several years, I mistakenly believed that the next level of learning is knowledge. BUT recently heard something that triggered me to realize that in order to move from information to knowledge, we need experience.
Knowledge is developed when information is applied. From that application, we can quickly see whether that information is correct or should be modified slightly to better fit the data that is observed.
Let me share an interesting example.
Machine learning is an area of interest to me. I first started working with it almost 30 years ago. As the world entered the Covid pandemic machine was used to learn what mattered in the treatments and what was not successful.
Machine learning failed to translate the data into information. It never even got to the knowledge level. WHAT!!!
You would think with a massive crisis and that AI/ML would lead the way to solutions. NOPE!!!
Why?
Well, the problem came in at the data level. From what I have read, people were collecting data BUT the data was not very descriptive and inclusive of all aspects of the virus.
There is an old computer science expression: Garbage IN Garbage OUT (GIGO). If the data that was collected did not adequately reflect the virus we were learning the wrong things and not meaningful to solving the issue.
This is where the experience addition is important. If after the data was processed into information there has to be experience(s). It is during experience where we work with derived information model AND new data to see if it does fit the real world of new data. Data and information can be manipulated and it might not reflect reality.
Recently, I saw a presentation that Andrew Ng, a leader in machine learning, gave in which he states that 90% of ML effort is on algorithms and 10% is on data. Maybe after the Covid experience, people must learn how to get better data. Maybe it should be 90% on data and 10% on algorithms?
What you reward you get!!! So researchers need to be rewarded if they focus on data instead of tweaking an algorithm to extract the last bit of juice out of it.
So get your data, process it to get information, let experience guide you to correct your beliefs, and ultimately arrive at knowledge.
I wish you well on your learning journey.
