September 29, 2002

Data, Information and Knowledge

This in short is the brief history of the universe. I am not writing this blog after being overly fascinated with the book with the title that sounds obscenely like the quip I just quipped. In fact I think I read that book a long long time ago. What I am writing this is because I have an alternate view of the universe. A view that breaks down everything into a point on the line, defined with ends of data and knowledge, with information lying in the middle. I know I am getting a little too abstract here, but then I hope things will be clearer to be as we go along.

When I used to study science, something struck me as very odd. Physics, especially of the variety that is normally taught in the high schools, breaks down the universe into two sections. The physical universe and the law that govern this universe. What struck me as odd, is the fact that god actually defined such a cute little dichotomy in his world. Just like we have data and instructions, male and female, good and bad, we have matter and laws. Okay matter, energy and all that dark crap too, but basically the tangibles and the intangibles. Okay, this is not also very true, but... Wow, this is tough, getting the definition right. But basically the problem with god and his universe boils down to this - how did he come up with something that exists, and then threw away a hell lot for us to discover. Why all this segregation? Why this duality? Why was matter there, for all of us to see, and the rest of the relationships, laws etcetera for us to discover?

But then think about it. What was matter. Ask someone in the dark ages, (dark ages NOT defined as the time before the computer) and their *ologists will tell you that it is nothing but a combination of air, water, earth, fire and something else. Somewhere down the time line, people will tell you that matter was made of unbreakable balls, called atomz. Then people went berserk. Matter was made of all sorts of strange. mystical and mythical substances, which incidentally no one can see, but ought to be there for matter to make sense.

So what was different in matter in the dark ages and now? Nothing. It is the same old matter burnt and forged into different shapes, but still the same old matter. What changed is information. The information known to man and this knowledge has changed the way people look at matter. If this information was not available, a lot of people in nagasaki would have been nth-generation residents, instead of what they are now. Matter has changed because of what matter is to different people. To the ordinary man, matter is nothing more than just earth and air. Hence what is important about matter is not matter itself, but information about matter. What we see as matter is nothing but the extract of the information conveyed to us by the various input devices.

We will now look at a totally different way of seeing the universe. A way in which there is no difference between the various units of matter, energy, ideas, minds and everyother thing in the universe. This unified way of looking at the universe is going to help us define the entire universe on a one dimensional scale rating information content. This will then give us a powerful way of dealing with many problems on a vastly simplified, unified methodology.

But before this we need to get some basic framework necessary. We postulate the existence of three different types of entities in this universe. The first is the Data Source or the DS. The Data source is characterised by the fact that it contains data. It owes its existence to the data it contains. There is no restriction on the data it contains. Of course we havent defined what data itself is. But we are deliberately not defining it, since it will be globally defined with the circumstance under view. And moreover, we cannot define it in isolation from other units underconsideration. Now the second entity we postulate about is the existence of the Data Acquirer or the DA. The Data Acquirer or the DA can query the DS for data through the use of what is known as a Data Transfer Medium or DTM.

Given these basic units, we define some terms. The first term we will be defining is the Data Completeness (DC) of an entity. DC is defined as the relative content of data of a particular kind in a particular Data Source. Hence DC is defined for a DS and Data Type. For example a DS has 100% DC about itself. Any DS can answer any question itself. So its DC is complete. Note that DC is independant of the query for data, or the way the query is designed, or the DA itself or the DTM for delivery. The actual response of the DS to a query is a function of the ability and capacity of the DTM and the DA.

This leads to an interesting and obvious statement. Any DS is 100% Data Complete with its own data. In fact, any entity, which is 100% DS with the data of any DS, is virtually indistinguishable from the DS itself. This is because the said entity can answer any question about the DS. This means that any DA cannot distinguish between the impersonating entity and the Data Source itself.

Now the DC itself does not give any powerful medium for expressing data relationships. Since the DC is fully defined with the data type and the DS, we define another term called the Relative Data Completeness or the RDC. RDC is defined as the relative compelteness of data given the DS, the DA and the DTM. For example, a still photograph has an RDC of close to 100% for the original static setup, given that the DA is just seeing them with just sight as the source of data input. The moment the DTM expands to include say touch, the photograph no longer has 100% RDC.

The RDC therefore gives a powerful medium to express the quality of data relationships between the DS, the DTM and the DA. We will dwell more on various examples for these terms in later posts.

Data is always handled in packets called observations. This observation is not the observation that is defined for an experiment. Observation is a taggable block of data. Observations differ from one another in their quality. Observations are generally substantiated by data. The amount of data represented by an observation is its relative richness. Richness of an observation is defined on a scale called the Linear Scale of Information or LSI. Data is one end of the LSI scale, while knowledge is the other extreme. Information is lying in between. Data are the small individual pieces of information, that border on indivisibility. Knowledge is completeness of knowledge. An entity which has a 100% of Data Completeness (DC) is perfectly knowledgable, and can infact replace the DS itself. An entity having an RDC (Relative Data Completeness) of 100% implies that for a particulat DTM and a DA, the entity appears to be the DS itself.

We will stop this round of definitions here. Check back for more data and information on these terms soon.

tada for now


September 24, 2002


I have this door that leads into the small balcony outside my room. The door is a brown wooden one, with a cast iron latch and a aluminium handle to pull it shut. It has a door stopped which is a wooden wedge hinged to the door jamb. When the door is open, this is swung to fit into the gap between the door and the jamb. Now what often happens is that sometimes the door stopper is in place and I have a class to go to. So i hold the aluminium handle and yank the door, when it fails to budge giving out a small crunch as the screws in the door hingers are strained out of their sockets. I mutter "oh shit!", and un-stop the door and try to close it. And the act of trying to forcibly shut the door misaligns it and it wont fit into the frame without a lot of pulling and grunting.

And I am sitting here writing about it.

Well, just outside the door is the balcony it leads to, (wow!!). It is about four feet deep. And sitting here, I can see the wall in great detail. The bottom of the wall has the floor skirting to prevent it from gettnig muddy. And this skirting is not not only muddy, but also wet and slimey from all the rain water that fills up the balcony. The floor is a similar color. But the wall is white. Guess that is why the skirting was provided in the first place.

The top of what wall is relatively clean. But the sides bear impressions of the dirt that flowed down the sides every rainy season, and the algae that grew on it after the rains. The most important thing about my balcony is that there is a tubelight right next to it. And I can sit outside at night and read books by the light of this light. And what is even more remarkable is that, this does not hurt my eyes when i am sleeping. And finally mosquitoes are miraculously not attracted to the light, even in the rainy season.

Outside my balcony is the outside. Some trees, some bushes, some undergrowth. Cool wind always finds my room in its path.

hmm.. just smell that.


September 16, 2002

Wish I was a Camera sometime

This is a line from a song by Bon Jovi. But I dont wish to capture just her beauty alone, I want to capture ALL beauty. And I just dont want to be any camera. I'd rather prefer if I was a Nikon Cam, with an assortment of lenses, and a stand at that same time. Otherwise, being a camera is just not worth it.

And, of course, I also wish I and my camera were invisible. But of course that would lead to problems with the photonics of the entire process of photography. Light would be free to go through my cam, and a shutter would not really help. But then, I really want to be invisible, and hope my reel becomes opaque just for the exposure time to catch the light, and it goes invisible again.

*Grin*. Okay, the reason I want to be invisible, is that I want to capture moments, as they happen. I dont want people to freeze up because I am there. I dont want birds to fly off. I dont want animals to give me glances and slink away.

That probably is the reason that I rather stick to photographing, dead things and other things that cannot run away from me. But I really want to be invisible.

Did you ever sit next to that road. And looked at the myraid colors and images flash past you. The sights and the sounds. The smells and the colors. The people in their hues and colors, going about their business. Businesses, varied, moving, and incredibly fascinating. And you wish, you were a camera sometimes.

Someone looked at some of the snaps I had shared, and had written to me expressing his opinion that I was a nature lover. Hmm, maybe, maybe not. I think not. I am not a nature lover or anything. I am a photolover. I am just photophyllic. Just dont care a damn, what those photons bounce off, as long as they differ in energy in patters that i like.

Art and science.. Or is that art and crap. you tell me.

Why is it that the cheap food is always the best tasting. If it has too much of oil, cholestrol, and is probably cooked in unhygienic surroundings. And it just tastes yummy!


September 14, 2002

long time no C

Listen to a group called "Jars of Clay". If you are one of those that loves ROCK. But dont mind the occasional strings. This is the group that will freak you out. Real kewl music, somewhat yucky but appropriate vocals, and the best of it all, acoustic riffs only. Sometimes a little other strings thrown in. Really good mixture.

What kind of a user are you? What do you use in those innumerable online forms that want you to describe your level of computer usage? Where do you fare on the four point scale - Beginner, Advanced, Expert, Guru. Actually how do you define where you stand? And how do you do that with respect to somethin as abstract as "computer usage"?

You know what I feel. When it comes to computers I think we are living in a dream world. I somehow have the feeling that I will just wake up, and all things that dont make sense to me suddenly wont exist anymore. Here is a sample of what baffles me. "What dont we have an introductory course to computers".

Wait. I know most of you would be really ready to click that small button on the right top corner of your browsers. Hang on for a sec. Think about it, there are no really introductory courses to computers. There are a lot of people out there, people who offer course, people who understand computers, who make the mistake of totally screwing up an introductory course. Who make the mistake of giving content that has little or no meaning to end users. Who dont respect the difference between an end user, and a budding programmer, or an entry level system analyst, or a rooky business administrator.

Most (okay, all the course that I have been exposed to, the most is only a disclaimer) of the courses I have seen suffer from one of the two problems.

  • The course has the wrong content for a focussed audience
  • The course, introduces the end user to the computer, not the computer to the end user

Take a course that is typically floated in academic institutions, CS101 Introduction to Computing. More often than not it will have some bit of electronics, some stuff about decimal and binary systems, then it goes on to programming logic and problem solving and generally end with a project in C. WHY??? The next course would be somewhere in the second year CS204 Fundamentals of OOPS. Again, WHY??? What is this course seeking to achieve. Just tell me how many people are actually enlightened by such a course. I personally dont think anyone would be. If OOP really tiltillated you, you would have read Bjarne Stroustrup anyway. So what is this course doing for you. Of course pedagogy is abysmal. Mostly you have professors who learnt OOPS when it was still an embryo, teaching you that course, who in a nutshell, suck.

Cut. Go to a beginner course in a professional Computer Academy. The course reads like: C/C++. Introduction to RDBMS, Internet technology, JAVA. Hey, hold on. The guy does not know what a bloody computer is for god's sake. What the hell will he use a database for?

Cut. Go to a Introductory seminar in a Corporate Place. C/C++, Word, Excel Functions, Macros... WHY? Hell, darling, your user does not know one error message from another. All errors are the same to him. Word is fine, but he is afraid of the computer dammit.

Cut. An introductory course in a Business Management Institute. Internet, eCommerce, SAP and other tools, Networking, XML, Web Protocols. Gurgle, gurgle aaah. Hey do you know what a nincompoop he is going to prove himself to be. The information you give him will be all that he is ever going to know regarding these topics. And to his dying day, he will assume that he knows computers. Do you have any idea about how much damage you are doing?

Will some course teach someone, who does not yet love computers, to love it. Will some course teach a person that a computer is an extremely logical unit, and that all you need to know is a few principles and everything will make sense to you.Will any one teach users what to expect from a machine and what not to. Will someone tell users the incredible beauty of computing, the power of C, the modular C++, the vastness of the Internet. Will someone break open a computer and show people, that inside it all, you just have some dumb wires. Will someone help the beginner, not feel threatened by the machine. Will someone help the beginner, not feel threatened by anyone who is not an beginner?

I sure do hope so. Will put out a 10 important things in a Introductory course soon...


September 08, 2002

Context Sandbox

This came to me when I saw some students carrying their CPUs to the presentation for a course. There is this course we have. Dealing with databases. And for that course the students have to do a project - a program. The platform could be anything, as long as it used databases for functioning - ASP, VB, VC++ whatever. And at the end of the term, the students taking that course have to make a presentation.

For the presentation, the students had to carry their CPUs to the professor. Why? Well basically, the architecture for the project required the students to connect to a remote database. And this was generally a painful process, considering that we are not doing Computer Science here, and most of the times, a final connection is establised through a lot of trial and error. By the time a connection is established and a connection string is finalized, a number of changes would have been done to the system that the student is working on, and the student would not be in a position to replicate the same on another machine.

Thus, this database project used to fail regularly, if the student just carried around the program as code, or as the executable (remember the DAOs and ADOs required were part of the OS) or in any other format. This almost forced the program to run only on the machine it has been written to run in. Thus we saw people carrying CPUs to and fro the prof's room for the presentation. The idea was that the all that needed to be added was power, and the program would run.

What do we have as part of a computer system. An application and data. Right? Wrong. There is also a context. The context is the executing environment of the application. Now in modern computing, this context is defined by the OS to an extent. And the context is realised by .dlls, APIs etc. The idea being that those entities not inherent to a particular application should be outsourced and be maintained by another party, or the OS. But look carefully, there are a lot of cases when third party tools are installed only for a particular program.

Lets look at some examples from the Windows world. Plugins into programs are one such set. Say Photoshop plugins, or Internet Explorer plugins, or Acrobat Reader plugins. For most of the scenarios, the application (Acrobat) and the data (.pdf file) alone would constitute the complete context. But for say some other scenarios, the plugin would also be a part of the context. Without it, having both the application and the data would be effectively useless.

Lets look at another example. Codecs. Say you have an AVI file. An AVI file normally can encode its video and audio streams in different formats and the application requires codecs to understand the two streams. Now what good is a great movie (data) on your machine (media player) without the Codec.

The above examples illustrate the need for an execution context in addition to the application and the data. Now lets look at what a context sandbox is.

A context sandbox is that minimum amount of information which will allow an action to be performed on a secondary machine, when the action is currently being performed as such on a primary machine.

There are some qualifications to be stated here:

  1. It is assumed that the primary and secondary machines are fundamentally capable of performing the action. In other words no definable context sandbox can exist for your washine machine to play your favourite movie.
  2. Information will be assumed to mean only that relating to software. Software will also be loosely defined as a sum of data and instructions. This means that information such as "Go get a life, buy another mp3 player" is not a context sandbox for a primary machine which is an mp3 player.
  3. Quality of performance is not an issue we will be dealing with here. Fundamental capability does not promise quality of performance.
  4. A machine is defined as the sum of all units that allow performance of a particular task. This includes hardware, software and any other environmental issues including power, temparature etc.
  5. Performance "as such" implies without change to the machine. Of course the machine being as described above.

So that is the idea. We will look more into ramifications of it in future posts.

Watched Memento today. Really kewl movie. This is the second time. Nothing new was learnt, but spent some time on the nuances of the amount of overlap the screenplay writer allowed between the scenes. Really well thought out.


September 06, 2002


\En*light"en*ment\, n. Act of enlightening, or the state of being enlightened or instructed.

Enlightenment is a completely themeable, highly configurable Window Manager for the X Window System, traditionally used in Unix environments. - That is from the homepage dedicated to the Window Manager for X.

But we are talking about the former - The state of being instructed. I like to think of learning as a passage from ignorance to enlightment. That is the only way learning should occur. Let me define 'enlightment'. When you learn something new, there is a time, when you dont know anything about it. And as you learn it you go into enlightenment. You know you are enlightened, when the topic is no longer strange to you. You know the basics of the topic. You know what rules are involved, and you really 'understand' those rules. The topic is no longer a representation for a cause-effect. You no longer understand the topic in a particular context, but understand the idea beyond the concept. That is enlightenment.

Let me explain. There are a lot of ways of learning

  1. Just learn that A gives rise to B
  2. Learn that class A gives rise to class B
  3. Learn all cases A which can give rise to corresponding cases of B
  4. Learn the reason why A gave rise to B
  5. Learn what happened
  6. Learn why something happened

The case (f) my friends is enlightenment. That is when you dont need to learn anything. The memory requirements are minimal. You learn a basic law of the universe. You have understood something that goes far beyond where you saw it first.
"Into this state, my teachers, let my brain awake..."

That is the problem with our learning system. We learn upto c, and then someone comes along and gives us smatterings of d and f in utterly in correlated chunks. Let that change, things will not be so tough after all.

Wishful thinking? mebbe, mebbe not.

Nothing goes according to plan, if it did, why would it still be called just a plan. It would be reality.


September 04, 2002

Points of View

Did you notice, that whenever you differed with someone, things never really go the way you imagined they would. You would have gone through the entire argument, and discussed it over and over again in your head. And it would have seemed so obvious then. The first word out of your mouth and you are on your own.

And moreover when you are giving a point of view, observe your audience carefully. The main question they will be airing is not - what? But why? Not, "what" is the opinion and what are its merits and demerits. Rather the focus will be on why you are taking that point of view. Sad, but true. So the next time you listen to a differing point of view in an argument, spend your time thinking about the what and try to let the why for a later date. Then look how much you will benefit and how many potential arguments will immediately peter out.

Gotta do some submissions in the next few days. Oh how I hate this!


September 03, 2002

No post yesterday

Not because I was busy or anything. In fact I did write a good post. And then I hit the post button, which immediately resulted in a 500 server error. Phut... The entire post - lost in electronic space. Funny right? We are building networks, to keep things under greater control, and make them work for us. And what is happening? Things get more mysterious and inexplicable. For example, where did my post go. Okay you will trace it to some web server. Then to some RAM space, which subsequently got de-allocated. So? Where did my post go?

Today, I made that heavenly drink again. Soft. Take some peppery mixture, and jet some aerated drink into it. It is heavenly, especially if the receptacle is an earthen pot.

We went to this place. It is called poilon. This is a dead giveaway about me for some 5000 people in this country. The rest of humanity wont know about it. Well, it is this place someway further down on the highway next to my Institute. It is a small, unkempt place. Basically an eating outlet - cheap and affordable. And the food there is incredible. You go there, and sit cross-legged on these wooden platform things. Then you order the stuff you want. The stove is almost next to you, and as the cook prepares your order, you can writhe in the sweet agony of the delicious auroma of your order. And when you are sufficiently famished, on the verge of mental breakdown, the food arrives. Piping hot and delicious.

You just cannot help stuffing yourself. Even as the tears pour down your cheeks due to the hot food. Then we made this heavenly drink. Oh god, it is almost as if you died and were standing in front of the pearly gates.

I just hope this post is posted and does not result in a 500 or a 404 and quietly comes as a 200. Amen


September 01, 2002

Sphere of eConsciousness

I had defined a term in my post on Tuesday, August 20, 2002. The term was Sphere of Perception. I had said that it is the sphere which a person identifies and understands. It defines the region from which inputs are used by people for learning.

There is a related term, I want to talk about. eConsciousness. Offline, your entire perception is defined by your senses. Your sight, hearing, feel in addition to smell and taste. So what are your online senses? What part of the online life are you attuned to? What is your source of information from the online world? To round it off, what part of your online presence are you attuned to? What is your eConsciousness?

We shall define eConsciousness loosely as that part of the online life you are attuned to. This basically consists of two parts. The first part is the part of the online life you actually know and identify. What exists on the World Wide Web. The answer to that question is the first part of eConsciousness. The second part pertains to that part of the online life, which you truly understand. What is the part of the life online you truly know. What part are you comfortable with. What part of the online life you know makes sense to you, and is not a source of fraud waiting to happen.

Why did I get this idea? Well, it goes like this. I got this message from SmileyGram for an E-card. This was basically from a friend of mine, who apparently had given my email address. I had to go and check out what was happenning there. So I checked the URL and pointed my browser to it. It took me to a page which needed 5 names and email addresses to show me the page. And nothing was optional. Of course i filled it with ****@****.com to get to the page.

I then mailed my friend asking her not to put my email in such forms again. I have been long enough on the web to have my name in a decent number of email databases. I really dont want another one getting hold of mine. The reply was along the lines that this was a 'good site' and that it 'just' needed 5 friend's names and the card is 'worth it'. My friend is showing remarkable eConsciousness of the first kind, not much of the second variety.

This I think is the case with a majority of the users. These are the users who click on the funlove messages. These are the people that download and run cute screensavers with trojans and backdoors. These are not people who are exactly alien to the online world. In fact they know of more free email sites, more free ecards sites than the first 3 pages of google. But they dont really understand the net. If we are to make the net a more secure place, it is this set of people we have to target. It is this set that must be told that not everything on the net is what it seems. Educate them, and we just might become safe in spite of all that Microsoft has planned for us.

One day I am going to develop an eConsciousness quotient.
Should help. Dont you think. tell me if you think so, or if you dont. [of course after demungle the email id]
Hmm, the winter is coming. Things are becoming cooler, and I have started skipping the baths. :)