Kosh: Building a mobile user experience for myself ;-) - I

I have been a smartphone user ever since I went mobile, my first phone being a Symbian Phone. Since then I have, on daily basis used newer version of Symbian, Android (up till Gingerbread), and now I am loving the beauty and elegance of Windows Phone 7 (Mango) on my Lumia 800. Intermittently, I have also dabbled with iOS and Bada. None, however come close to the usability that Windows Phone currently provides. For those of you, who still don't 'get' the Metro UI, this post might enlighten a bit: http://spillwaybrain.wordpress.com/2011/11/10/why-windows/
My primary requirement from a Smartphone has been programmability, preferably on the phone itself. With Symbian, I really enjoyed the pys60 (http://sites.google.com/site/tovganesh/s60) and now with Windows Phone, I am having fun with TouchDevelop (https://www.touchdevelop.com/wblh), more on this in subsequent posts. With Android though, I just couldn’t get connected the same way.
While Windows Phone is a great OS, the current price point makes it unaffordable to large section of people. This may change over time, particularly in combination with Nokia. On the other hand, Samsung, with its Bada OS, plans to build a ‘smartphone for everyone’. However, with the current programming model for Bada, particularly the use of C++; doesn’t really go well with the philosophy. Because, to me a smartphone should not only be programmable by a professional third party, but also as easily by the owner of the device; just as is the case with the PC. When you have this joy of programming your own device, you kind of connect to the device for a long attachment, not just for period of your current contract (anyone remembers the BBC Micro?).

The Phone
Most smartphone designs today seem to forget that it is actually a phone. So making a phone call on a smartphone is typically much slower than on a very basic Nokia device. You can't just start keying in the number to give a call. You have to open the Phone App and then give a call. And in some designs you even have to press one extra button to bring up the dialer app! Oops that is a complicated phone, but we still like to call it smartphone, for some reason.
So the smartness comes from the fact that it is able to do a whole lot of other things than just being a phone, not that it is easier to make a phone call from it. It serves as a communication hub. It basically tries to provide a complete mobile computing experience in your pocket. Over time, probably we will use the phone component in it a lot lesser, and then it would be more technical to call it a smartmobile.
Another drawback of most smartphones today is the battery life: it is simply unacceptable. Though my current Nokia Lumia 800 easily lasts for a day and half of normal usage (always on data, few calls and text messages, occasional games, about half hour of GPS usage - for sports tracker), for it to be usable by a large majority of people in a nation like India, it needs to run on solar power or an alternate form of energy (body energy, crank shaft, anybody?). Currently, though I can see Solar energy as the only viable solution. This could be coupled with a crank shaft enabled charger, though my experience with a crank shaft device (Philips radio), has not been that great : probably it had a design flaw.  For most of the mobile devices today, the display is one of the components that consumes the maximum amount of power. Without the display these devices could go on easily for more than a week. Take for instance the Kindle, it can easily go for about 10 or more days, even with the 3G radio on; with the radio off it lasts for more than a month. The one major difference is the screen: e-ink. Although the e-ink looks fine for a book reader, for a mobile device it will not work out. Options would be to use color e-ink or PixelQi screen. The recent demonstration by the OLPC project [http://www.youtube.com/watch?v=ITHNbOrPQyM] shows some hope, but a lot of work needs to be done to make it to a device that can be held in your hands.
To design the right hardware would be a challenge. At this moment I am not very clear about what this hardware would be, so I will use followup posts to put in my thoughts on the hardware side.

The UI
So, I want a smartmobile, whose primary functions is to stay connected with the people you care about: friends, family and co-workers. The Metro UI, which I pretty much like, tries to do this with what is called as a People's Hub. This is a well though out piece of UI, which threads all your contacts, communication into a single place. This also connects to the social media, and brings up the latest updates of your friends.
Taking cue from this, and observing how I use my mobile device, I think there are three most important things I do with a mobile device: Keep in touch with people, Search for information and use Applications.

So let me introduce to you : Kosh (Sanskrit: a collection / repository)
It is a collection of things I do:

• People Kosh : The collection of all tools allowing one to connect to people. The Kosh UI is always accessible from anywhere (if using a full screen application), by using a Swipe down gesture. People Kosh needs only three main components: Phone (which includes VOIP / video calls), Message (which includes SMS and Email) and Social updates (a quick way to be in touch with people on your social networks). The Phone and Message Apps are also 'live tiles'. Unlike Windows Phone 7, they will display additional information, such as from whom you missed the last call, last line from the text message you received recently etc.

• Search Kosh: The collection of all search tools. This will be central place to search anything (well probably some things in life are still better searched on your own ;) ) : either on the phone, or the Internet. The search can be either text based [input: on-screen keyboard], voice based [input: microphone, audio file] (this will not be just voice recognition, but should be localized, and probably should integrate music search, radio search etc.) and visual [image, camera] (scan image, photo, QR code, bar code and probably video clip).

• Apps Kosh: The collection of all apps. Device status like remaining battery, Network connectivity are just another apps. One need not have integrated UI for the same. Each icon acts as a means to launch the application. By default the icons are alphabetically organized (as grid of or as a list). Application icons may display notifications. Notifications can be in the form of text or image or a combination of the two, and are displayed alternatively or along with the default icon. Icons with notifications automatically move up and have a glow surrounding them. Once the user taps the apps with notification to open, the icons are automatically placed in their usual location.
  To switch between Apps, use three finger pinch. This will show a list of 'active apps' with end of the list being signified by a shadow on the edge. An active app is not the same as a background app : in fact, on a constrained device it would not be really a good idea. An active app here, is the one that has registered to the system for a background notification. Alternatively the only app that is available 'in memory' is the foreground application. The applications that require background processing (play music, for instance) will be handled by a system process instead.

The Architecture
Well, I want to build the full stack. That is because whatever is currently available is probably not going to work for me. It is not only important to get the kernel done right, but also the rest of the components done well to make for a great user experience (UE). The diagram below shows the most important components of this stack.

The Kosh system will be build on a mirco-kernel architecture. This would mean the Linux (or Android / Palm OS ) kernel would not be a good starting point. A QNX like kernel would be the way to go. The other components are the telephony and network layer, specifically for handling the communication needs. The system process for notification and other management purposes. The Kosh API layer for providing APIs for the UI layer as well as writing third party applications.

The Kosh API layer will have the above major components. I think for all that I outlined about, these set of broad APIs would be just enogh to start with. The UI layer will be build using these APIs and a WebKit based (or WebKit like) rendering engine. This essentially means all user interface layer will be build on HTML 5+ and JavaScript. Consequently, the Kosh APIs will be exposed as JavaScript. Also all the third party apps will be written using HTML and JavaScript. This is a philosophy quite akin to webOS. Essentially this means that EnyoJS might be a good starting point for looking at API design. Another project to look at is Mozzila B2G, the  Boot to Gecko initiative.

A carrier independent network
We are over reliant on the 'carrier network' for all the communication today. Though 'carrier networks' will not disappear overnight, it would be good to have a way of communication that does not rely on a centralizer carrier.That would allow for a medium of communication which is not overlay regulated. For emergency needs though, a carrier or satellite based solution is still required. Irrespective of this, for me, communication among humans should ideally be free, that is what takes the race ahead. At the moment though this is not exactly the case. There is the Internet, that was supposed to be a free medium, but due to one component: the DNS, it could be unfairly controlled. TCP does allow peer-to-peer communication, with no centralized control, but again for people to 'find' each other, we need a kind of directory service. Skype for instance uses peer-to-peer communication, but requires a central authenticating authority. A few years ago, Nokia, experimented with peer-to-peer, proximity communication, using an application called Sensor. The Sensor application (still available for old Symbian), used the Bluetooth as a means of communicating with people in proximity (instead of directly talking to each other!). The Sensor app would host a mobile page of you (like your facebook page), people could see this from their apps and if they like, send message to you, and probably become friends. Sounded like a pretty good idea, but somehow did not catchup, and the project seems to be abandoned altogether, at least for now.

I think, at least for local communication needs, one can setup a peer-to-peer communication system that is independent of any infrastructure. Every device in this system behaves as a carrier. Consider that everyone owns such a device. Also, in a modern city, I think it would be safe to assume that every person will find at least another person with in 10 meters radius (this is just a hypothesis right now, a proof for this would be amazing). All the devices in this system will have a unique number, and will be owned by a person. Every device will have a public and private key for the purpose of security. Now consider that a person wants to send a message to his mate. The way this would work out is as follows:

• The sender knows the public key and the device identifier of the receiver. For this to happen, both the sender and receiver should have met physically and exchanged this information once.
• A message packet is prepared with the device identifier, and the content encrypted using public key.
• In addition, the message packet will contain a field for signifying the importance along with a hop counter. At the source, the importance field will have the highest value, while the hop counter will be zero. Every second hop a packet makes the importance is decremented by one, where as the hop counter is incremented with each hop.
• The message is then broadcasted to all the nearby (say with in 10 meters radius) devices.
• If any of these devices is the intended device, the process completes. There still might be lingering packets in the network carrying the original message, these will eventually die as their importance decreases.
• If the importance field of the message packet reaches a negative value, it is not re-broadcasted, else it is re-broadcasted.

The above is quite akin to the way messages were transmitted in olden days (that were not from Kings). I think this should work wonderfully if the density of the device is sufficient, though at this point I don't have a proof of what this sufficient number should be.
One drawback of this system is power requirement. I am not sure a device that would handle this amount of traffic can last for long, given today's battery technology: the radio (apart from the display) is a major power guzzler.

Alternative positioning
While the GPS works great where I live, and I am actually pleasantly surprised by the accuracy of Nokia Drive app on my Lumia, there is a need for a cheaper solution that will work even if the satellites are out of visibility. Nokia Lumia and most other modern smartphones supports A-GPS, which augments the satellite positioning with other information such as cellular tower positions, wifi hotspots and the IP address of your device to give pretty accurate positioning information.
Another approach would be to use landmarks as a way of positioning. This would basically involve the use camera and some image processing. The device will store information on the known landmarks and their positions. The stream shot coming from the camera will be mapped on to the available landmarks and approximate position calculated. A system like thing, might not be good for turn-by-turn navigation, but will be useful if your are walking or cycling and you want to roughly know your current location. Again, this is not a substitute for GPS, which also provides altitude information, but then for day-to-day use this might just work out.

Programming Model and On-device programming
Kosh will be using HTML + JavaScript for user level programs. Any hardware level code will be written using pure C.
The most challenging aspect, though is to create an On-device programming environment that is easy to use, but also powerful to take full advantage of the capabilities of the device. In my earlier post on Nokia Lumia, I had mentioned about TouchDevelop from Microsoft Research. This is terrific on-device programming environment and I have not found anything yet that comes even close to this. This is one environment that makes me love the Lumia (to see stuff I am up to using TouchDevelop, visit: https://www.touchdevelop.com/wblh). 
To make Kosh really interesting an on-device programming environment akin to TouchDevelop must be built.


References
[1] http://spillwaybrain.wordpress.com/2011/11/10/why-windows/
[2] http://enyojs.com/
[3] http://www.eink.com/display_products_triton.html
[4] http://www.pixelqi.com/products
[5] http://www.qnx.com/products/neutrino-rtos/index.html
[6] http://www.webkit.org/
[7] Wiki on Nokia Sensor: http://en.wikipedia.org/wiki/Nokia_Sensor
[8] Mozilla Mobile web OS: https://wiki.mozilla.org/B2G

Followup
At this point, I only have idea. Not a single line of code to show. But to start it up, I had to write this down. This is just a beginning of the things to come, and hopefully you will love it too :)

Fine Print
The ideas, text and graphics in this post are exclusively created and owned by the author. The templates for few icons in the post are taken from a Google image search, they were then suitably modified for my needs. All the above content, and any followup posts with the label "kosh" is henceforth made available under Creative Commons License (http://creativecommons.org/licenses/by/3.0/) and may be attributed as : Kosh, by V. Ganesh.
The text also contains references to product names, and are trademarks of respective organizations. They are used in the text for information purpose only.

An inside look at one of the first popular smartphones: Nokia 6600

The first ever mobile phone I owned was in fact a Smartphone: Nokia 6600. This was one of the best phones available at one time, in terms of cpu, memory and availability of installable 3rd party applications. It even had a modern day browser: Opera Mobile. Was just a perfect piece of hardware: but lacked 3G. And then the advent of iPhone just made it look too lame. I always wanted to have a look inside this smartphone to see what chips make it tick. So, finally I took apart the circuit board of 6600. It has a 104MHz ARM processor, with onboard RAM of 10MB. There is also internal storage chips which I guess is either 16MB or 32MB. Pretty low specs compared to what I have on my Lumia 800 (which has almost comparable hardware to my Asus t91mt – except on the RAM front).

A photo … and a national day that two nations share

It is republic day in India again. And I was pleasantly greeted by this tri-color on my Lumia this morning. Happy Republic Day!
Today is also the Australia Day,  so warm wishes to my Oz friends. I remember, discovering this for the first time few years ago when I was in Oz, that we share the same day as our national day. It was also the first time I was out of India on a republic day, but knowing that the other land also shared a national day, made it sweet .

|| वसुधैव कुतुंबकं || ( vasudhaiv kutumbakam )

A test post from Nokia Lumia

This is just a test post from my Nokia Lumia. Just to say that it works great. And the onscreen keyboard is just too good even for writing a long enogh post.
Just loving this :)

Back to Nokia, with a Lumia!

Well yes, somehow I convinced (rather spoiled) myself into buying this device. I was just to frustrated with my current Android device and I had no plans on falling back on Nokia E51 (the Symbian device that I used before). Well in the process I actually cleared up my digital inventory: essentially sold off my laptop (to get some funds ). So now I am reduced to this stunning new Nokia Lumia, my Asus t91mt and the regular desktop.
There was good reason to do that. I seem to be carrying a lot of “digital burden” with me. Particularly the laptop. At home I use Windows on desktop (with Ubuntu running as virtual machine). And when my desktop is being used by my parents, I use my Asus t91mt. At work I almost exclusively use Linux. I need a Windows machine mainly to take notes (I use OneNote) and if I need to make presentations or use Word. Since I don’t have a Windows machine at work, I mostly tried to use my personal laptop for this purpose. I also tried using Google docs, but it simply doesn’t meet my needs. And carrying the laptop was a pain for everyday travel. I wanted my phone instead to do the task, but it simply wouldn’t work the way I wanted. Moreover, it was simply problematic : the more I tried to use it, the more it crashed. Eventually, I found one smartphone that could do it all: the elegantly build Nokia Lumia with Windows Mango OS, and Office App built right into the device.

On the face: Beautiful, and Wow
Nokia knows how to build the hardware that blends seamlessly with the software. The Lumia 800 (I still think Nokia should not have put that number and just made one product with the name Lumia), has a stunningly beautiful, unibody polycarbonate casing with curved AMOLED corning gorilla glass display. To sum up all the jargon: its a beautifully designed solid product, with a soul in it.

Even the power adapter is nicely designed. It is slim as a plug, and you don’t even feel its an adapter. My only complaint is that it should have been black for a black phone, or they should have made a white Lumia.
And then there is this wow factor. The Windows Phone Mango adds the needed sweetness on the top of that wow factor. The Metro UI is a refreshing break from static icons of iOS and Android. Though one may say that Widgets do the same thing as Live tiles : it is not. Widgets are battery hog on Android, Live tiles are not.

Below the skin: pleasantly useful and gets the work done
A lot has been said about the “gets the work done” for Windows Phone and I can say this again. I have all that I need out-of-box. It connects to GMail, Hotmail and Yahoo without the need for any app download. Twitter and Facebook is supported on the phone : no apps needed if you are happy with basic functionality.
I use OneNote a lot, and the integrated Office Hub is a treat of usability for me. This is kind of “must have” functionality for me and Windows Phone does it just right.
The other application I use a lot is Maps. Although the Lumia comes with Nokia maps and you can download a third-party Google maps app from Windows Phone market place, I miss one feature: ability to search for public transport routes. The Nokia maps seems to support this, though it doesn’t seem to work in the place I live.
And most importantly: the device has no lag even when using it with loads of apps or browsing a number of sites. This is remarkably better than any Android device that I have used so far.


Apps and all
Generally all the reviews on the Internet these days are all praise for Windows Phone, except one thing: the selection of Apps. After using the windows phone marketplace, I don’t think I fully agree. I found my most used app on the other platform in the marketplace. Moreover, these are more beautifully designed than the once available on other platform. Sure the number of Apps in marketplace is smaller, but definitely not the quality, they are simply superior.
Not everything is rosy though. Two main sour points are: Skype is entirely missing (though I hope this will soon get resolved as now Microsoft itself has purchased the company) and Adobe reader (while on all other platforms this has been updated to X, marketplace still has the older 9.0 version .. thats lame).

The Camera

The camera performance is reasonable for my needs to do casual photography. For serious needs I do have a DSLR. Of-course it is just unfair to compare this camera with my DSLR for quality, but Nokia could probably have done a better job at handling scenes with low light conditions. I also could not find a way to correct for “red-eyes” within the Camera application.  There is no auto-panorama on the default camera app, but you can get an App from the marketplace that does a pretty good job for this.

Sound and Music
Zune player on the phone is a nicely integrated app. The “music+videos” hub really makes it interesting as it not only integrates the radio, but also other online music services and related apps.
The speaker quality is what I would term as OK. The bundled headset is of good quality, but as with others, it slips from my ears (they just don’t get it that I am a good listener ).
On the Nokia Support site there were many complaints regarding sound quality when the headset is attached (either supplied or any other headset with 3.5mm connector). What I found is that it actually depends on the quality of the music file you are playing. With good quality mp3 files (the ones that I created from my own collection of CDs) had excellent response (even at the highest volume). This is not simply the case with another mp3 file I created from a cassette tape. It actually sounded better on my PC, but on the device it had this cracking sound as I increased the volume. This prompts me to think that this is more of an issue with the software that does the decoding. Hopefully a future software update will fix this.

Battery life
My usage of this phone has been quite heavy till now. So I can’t really judge the battery life in normal use. That said I got about 8 hours of continuous use with Wifi on. I will update this post as I get more idea of battery life on daily use basis.

[Update 1: I am still testing the battery life. But my normal use seems to last for about 24 hrs or more. Nokia also replaced my phone (without even me asking!), so I am actually happy :). Today I had put on the WiFi most of the time as I was writing scripts using TouchDevelop (see below). The battery lasted for about 8 hrs, that is sort of consistant with my heavy usage as observed earlier. ]

A new way to develop: TouchDevelop

My first ever phone was a Nokia 6600 (here) and then the Nokia E51 (here). After this I had a brief stint with Android platform (here). My primary requirement from any device has always been: programmability, apart from usability. I have been always hooked to the Nokais from day 1 because of the amazing programmability of these devices. I build a complete Quantum Chemistry program (mobihf : http://sites.google.com/site/tovganesh/s60) on it and even presented a paper on this work! I could not say the same for the Android platform, somehow I have been disappointed even before I used the platform (here and here). Although Android has an in device programming project (http://code.google.com/p/android-scripting/), it is not actually “officially supported”. Neither I got around to use this properly on my android device.
With Windows Phone, though, Microsoft Research has done a great job by releasing TouchDevelop (https://www.touchdevelop.com/help and http://research.microsoft.com/en-us/projects/touchdevelop/), a new way to program the device without a PC. On my Nokia Lumia, I was up and writing a simple graph plot script in not more than 15 mins (see this: https://www.touchdevelop.com/wblh). Although, I already have Windows Phone SDK installed on my desktop, I find programming on the device very interesting (as I have said this here and various other posts here), and is the only way to break from being dependent on the PC. This also gives you a feeling that you have complete control of the device and that you are not dependent on any third-party to get something done exactly the way you want it.
I think, Microsoft Research, with its TouchDevelop, has shown a way that this could be done. It may not be perfect, but this is probably the direction that we should take. Hope that this gets carried over to the forthcoming “Windows 8”, which would be quite a radical development.
In forthcoming posts, I hope to put in more efforts to cover TouchDevelop as I get more familiar with it and write more “fun” scripts .

Connecting to the PC : Zune

This is very simple. And as I have been already using the Zune player, I had no problems with connecting the device to the PC. Compared to Android (or Symbian) however, there is no way to expose the device storage as a filesystem on the PC. The Windows Phone is handled by Zune just the way iOS devices are handled by the iTunes player.

Nokia care: has it improved?
I had complained about my previous stint with Nokia Care (here). I am not sure of the current status. Though I must say that in-showroom experience of buying the Lumia was fantastic : very friendly and knowledgeable people.

In the end : would I recommend this?
Definitely. The major deciding factor for many would be the price. It would however offset if you look at it as a complete mobile computing device rather than just a phone, which is what it really is.

Changing times with the Rupee and the Paisa …

Was thrilled to see these new coins with the new rupee marking Also in the picture is the one paisa coin that is my constant companion, even though it was abandoned long before I was born. Hope that we can see the value of that one paisa grow up some day
Enjoy the weekend !

webOS goes opensource :)

HP is making the webOS opensource and it intends to make it "pure opensource" as opposed to the "other" controlling project. The press release of the announcement is here: http://www.hp.com/hpinfo/newsroom/press/2011/111209xa.html?mtxs=rss-corp-news and http://developer.palm.com/blog/2011/12/open-source/

Hope this serves as a viable alternative to all the "corporate" managed mobile OSes. I would love to see a homebrew, DIY community sprung around this and create more interesting ecosystem around it with least central control.

Just painting...

Mathematics add-in for OneNote

Okay this one is not only cool but very usable tool from Microsoft. If you use OneNote / Word and if you are into Science, you might find this tool very valuable. The mathematics add-in for Word / OneNote is available from Microsoft at: http://www.microsoft.com/download/en/details.aspx?id=17786

The tool combined with Tablet PC features (on my Asus t91) is quite a fun and addictive

Insanely helpful error message

Ofcourse, I don't see any harmony in the exception thrown !! ;)

(PS. Nokia, eagerly waiting to know when the WP7 devices are heading to India?)
(Update: So looks like, mostly probably Nokia WP7 is landing in India by the end of the year, ref: http://windowsteamblog.com/windows_phone/b/windowsphone/archive/2011/10/26/nokia-s-first-two-windows-phones-are-here-and-they-re-awesome.aspx. And in most probablity, I am gonna switch to one of the Lumia devices :) )

Steve, you live on!

http://www.apple.com/stevejobs/

 

qusoware: Quantum Software on their way

Until now if you wanted to try out any of the algorithms based on quantum computing framework, you would basically need to reconfigure the hardware,  presumably with an expert experimentalist. This however is probably poised to change with the introduction of quantum memory:http://www.technologyreview.in/computing/38495/. This essentially brings the quantum computing devices to the realm of general purpose computing architecture, the thing that is most common in todays processors.
On a lighter note we will might soon have qusoware (my pun for software that runs on a quantum computer), probably something like QComBasic 1.0 from someone like Microsoft ;)

Upgrade to MeTA Studio : full jython integration, first steps of jchempaint integration

A new upgrade to MeTA Studio is now available for download. The current version is 2.0.27082011 and the update is available from usual place: http://code.google.com/p/metastudio/

Changes in this version:
- Updated smackx libraries, GTalk integration works again!
- Fully supported Jython integration into the IDE

  - Initial implementation of JChempaint integration, more improvements on the way

  - Other improvements: surface normals, improved GTalk widget, bug fixes and improvements in federation framework, issues with .mar handling fixed, new APIs for managing workflows (no UI integration yet).
 - I also welcome javadba, who recently joined the project and plans to contribute unit tests for the parallel framework in the MeTA Studio and in the process improve it.

With this release, MeTA Studio SVN has 500+ commits (http://code.google.com/p/metastudio/updates/list). Though the development on MeTA Studio started quite some time back (2003-04 for version 2.0.x), it was only made opensource in 2009. Just interesting to look back ;)

Enjoy!

(PS: as you might notice from SVN logs, I unsuccessfully tried porting the project to JDK 7, which was recently released. I am almost sure that the problem is with a bug in the current JDK release. Will be investigating and may eventually migrate to the new JDK release once it becomes stable enough.)

Hello Kinect : Speech recognition

If you have ever used speech recognition built into recent version of Windows, you would know of all the troubles it has. It is good, but not good enough to make you want it use on everyday basis. It is good when it works, and painful other times. Handwriting recognition or merely typing is much less annoying to get your real work done. Most of the trouble comes when you are in a room where other people are also talking. The engine gets confused.
With Kinect, however, things are better as the speech recognizer also uses additional inputs from the camera multi-array microphone to estimate the source of sound to recognize. Which pretty much makes speech recognition quite interesting.
So what all do we need to identify speech using the Kinect SDK?
First of all you need to add Microsoft.Speech.dll to your project, and them make the following two imports:

using Microsoft.Speech.AudioFormat;
using Microsoft.Speech.Recognition;

All of the speech recognition stuff is then essentially handled by the classes: KinectAudioSource and SpeechRecognitionEngine. The later is part of Microsoft speech API and provides a generalized framework for speech recognition.

private KinectAudioSource kinectSource;
private SpeechRecognitionEngine sre;

RecognizerInfo ri = SpeechRecognitionEngine.InstalledRecognizers().Where(
r => r.Id == RecognizerId).FirstOrDefault();

if (ri == null) return;
sre = new SpeechRecognitionEngine(ri.Id);
var helloChoice = new Choices();
helloChoice.Add("hello");
helloChoice.Add("kinect");
var gb = new GrammarBuilder();
gb.Append(helloChoice);
var g = new Grammar(gb);
sre.LoadGrammar(g);
sre.SpeechRecognized += sre_SpeechRecognized;
sre.SpeechHypothesized += sre_SpeechHypothesized;
sre.SpeechRecognitionRejected += new EventHandler(sre_SpeechRecognitionRejected);
var t = new Thread(StartKinectAudioStream);
t.Start();

For the speech recognizer to work correctly, you need to provide words that need to identified. These are handled by constructing a 'grammer' for the same. In the above code we construct a simple grammar to recognizer only two words 'hello' and 'kinect'. Next we register event handlers, which are 'callbacks' when the SpeechRecognitionEngine recognizes (or does not) something that is spoken.
After this we open the Kinect's audio stream and start listening to it in a different thread.

The body of StartKinectAudioStream() function is as follows:

kinectSource = new KinectAudioSource();
kinectSource.SystemMode = SystemMode.OptibeamArrayOnly;
kinectSource.FeatureMode = true;
kinectSource.AutomaticGainControl = false;
kinectSource.MicArrayMode = MicArrayMode.MicArrayAdaptiveBeam;
var kinectStream = kinectSource.Start();
sre.SetInputToAudioStream(kinectStream, new SpeechAudioFormatInfo(
                                               EncodingFormat.Pcm, 16000, 16, 1,
                                               32000, 2, null));
sre.RecognizeAsync(RecognizeMode.Multiple);

The code above basically tries to construct a beam for each person recognized by Kinect (skeletal tracker).

Finally, the signature of event handlers for speech recognizer are as follows:

void sre_SpeechRecognitionRejected(object sender, SpeechRecognitionRejectedEventArgs e)
void sre_SpeechHypothesized(object sender, SpeechHypothesizedEventArgs e)
void sre_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
   Console.Write("\rSpeech Recognized: \t{0}", e.Result.Text);
   lastRecognizedWord = e.Result.Text;
}

Here a short video:

More to come soon :)

Google + Motorola = Googarola !

Here we go, Googarola (http://googleblog.blogspot.com/2011/08/supercharging-android-google-to-acquire.html)! Google has announced that it would be acquiring Motorola mobility, the unit of Motorola that build the Droids. Essentially, Google has stopped being software only company, by buying a major Android handset manufacturer. This is a quite smart move. From the official blogpost, it is quite apparent the motive of this buy: Patents. Lately, Android handset manufacturer have been hit by patents from all sides, some of which may be ligit, but overall they simply seem to seve as a way to scare companies away from using Android alltogether. With Motorola acquisition, Google gets its hand on a large pool of mobile patents, and at the same time also weakens its complaint againt Apple+Mircosoft et. al. to the US DoJ for the rescent issue of Nortel patents buyout.
Personally, I feel this is a good move in view of customers. Hopefully, the mobile space has room for atleast about 5 different ecosystems which are sustainable as against only 2 players in case of PCs. Google can now have more control over the hardware and at the same time will have less worry of any patent litigation. Also, I think this move is good for other Android handset manufactures, as now Google can essentially be more helpful to them when it comes to patent litigations. What actually worries me is if the culture of an 80 year old company will jell well with an yet young Google.
In view of this acquisition, one more thing seems to be clear: the reason why Nokia didn't go with Android. I guess, with Motorola acquisition in mind, Google obviously couldn't think of giving Nokia enough room for customization of Andoid.

Things in mobile space are getting very interesting. May be another major cosolidation is underway... like the 'unlikely' acquition of Nokia by Microsoft? Although I am highly skeptical of such an event.

Hello Kinect!

So, finally I gave up all the resistance to avoid buying this cool new stuff and ordered a piece for myself from http://www.flipkart.com/. The unit arrived last week, but had to spent the weekend rearranging my living room so that I could get enough space to use my Kinect. Plugging it to my PC and using the drivers provided with Microsoft Kinect SDK was straightforward. I did initially face a problem with the driver not installing properly, but quickly figured it out that this was because the Kinect was not plugged into a root USB port.

My intention of getting Kinect was not to play games, but to play with programming it. Although I would happily take an XBox, if you gift me one ;-)
This post and subsequent posts on Kindle on this blog will tell my experience of programming on Kindle. As a first step, I ensured that the samples provided with the SDK work well. Next, I installed Visual Studio Express 2010 (available for free here: http://www.microsoft.com/express).
I chose C# (C-Sharp, may be they should call it C-Dumb :P) as my programming language for Kinect. Jokes apart, I have very little experience using C#, mostly using Java or C++ (left using Fortran on day-to-day basis 2 years ago!). Any how, I found C# to be quite neatly designed language and easy to learn particularly if you come from Java or C++ background. If you come from C++, you are sure to enjoy some freshness that Java brought to object oriented programming.
So considering you have some idea to program in C#, writing a 'Hello Kinect' is relatively easy. First ensure that you have added Microsoft.Research.Kinect.dll as an external dependency to the Visual Studio project you create.

Next is to import the Kinect APIs:
using Microsoft.Research.Kinect.Nui;
All the initialization of Kinect NUI (Natural User Interface) is handled using the Runtime class.
Runtime nui = new Runtime();
nui.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | 
               RuntimeOptions.UseSkeletalTracking |
               RuntimeOptions.UseColor);

Next we open the Video and Depth streams of Kinect:
nui.VideoStream.Open(ImageStreamType.Video, 2,
                     ImageResolution.Resolution640x480,
                     ImageType.Color);
nui.DepthStream.Open(ImageStreamType.Depth, 2,
                     ImageResolution.Resolution320x240, 
                     ImageType.DepthAndPlayerIndex);
Note that the current Kinect hardware only support VGA resolution (max) for video stream.
When a data frame is available for processing on Kinect, the driver sends a notification to the application. In C# this is handled by registering an appropriate even handler as follows:
nui.DepthFrameReady += 
new EventHandler<ImageFrameReadyEventArgs>(nui_DepthFrameReady);
nui.SkeletonFrameReady += 
new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);

The signatures of the event handlers look as below:
void nui_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
void nui_SkeletonFrameReady(object sender, 
                            SkeletonFrameReadyEventArgs e)

Now, I have managed to see how to get the Skeletal data easily:
SkeletonFrame skeletonFrame = e.SkeletonFrame;
foreach (SkeletonData data in skeletonFrame.Skeletons)
{                
       foreach (Joint joint in data.Joints)
       {
              // transform and plot joint.Position
       }
}
In the end uninitialize:
nui.Uninitialize();

So here is a video of my ‘dot avatar’) in action

My ‘dot avatar’, the other ‘dot avatar’ is my dad in background. Kinect SDK for Windows 7 at the moment allows tracking of only two people.

Next up, I need to figure out how exactly to use depth data as well as handle audio. Hopefully a post for next weekend

Have fun!

A need for sincere debate on the energy issue in India

When was the last time there was a debate regarding future of energy resources in India?
Many would want to say ... um .. well when was it? When Indian Parliament was trying to get a nuclear deal signed with the USA? Or just last month when there was a fuel hike?
I have a strong feeling that none of these debates were sincere. None of them had anything visionary coming out of it. I always felt that there were more emotional and monetary reasons behind any decision involved towards the future of energy resources in India.
Everyone at the decision making level today seems to make statements such as, "Nuclear and thermal energy is the future for power needs of India". Seriously? Did we ever think if this is sustainable? How did we come to that 'conclusion'? 
To understand this let us go to the root of the issue. Bharat, traditionally has been an economy that always emphasized upon sustainable villages. The villages were always as resourceful as the cities. When the Colonial rule came to Bharat, they got with them the Industrial culture in the West and India was born. Industrialization was good, it brought the so-called 'progress'. There was one differentiating factor though which was not so good. It promoted an economy of unbalance. Of all the unbalances, the unbalance of a region or territory is what made things even sorrow. This is what resulted in the formation of 'cities' of modern time. Instead of decentralization, we had concentrated hubs that had all the amenities. And all these were completely absent in other places. This is what prompted large scale migrations from villages to 'cities', which keep on getting bigger and bigger. But how long will this expansion last is anyone's guess.

A case for decentralization
We can’t keep expanding cities. It will surely crumble at some point. We already keep seeing signs of this. All that the city administration tries to do is find workaround.
A good example of how not to expand a city is the one that I live in: Pune. This city is full of malls. So many of them and so close. And each of them suck power like crazy. There seems to be no full stop in growing number of malls. I can count like four big malls in my small suburb of Viman Nagar. And one of the malls even proclaim to be biggest in India! So you can easily imagine its energy needs. Add to this the fact that the city houses two of the most powerful supercomputers in the country, which also figure in the top 500 list. You can easily get where this is all headed.
When Mahatma Gandhi said we need self-sufficient villages and not big cities, he had point and a vision to see ahead. We just seem to have forgotten that, our vision blinded by the glitz of big malls. At the same time people migrate to cities because villages are under-developed, have rigid social structure and very limited opportunity for ‘growth’. The form of decentralization that Gandhiji advocated once seems to have been completely lost. No one cares about it. Heck none of the political class even speak of it, most are simply concerned with providing subsidies or ‘rojgar’ schemes, which most of times are pointless and foster corruption.

Bad Science and Why no investment in future, the renewable energy?
A few years ago a group of people from a wind energy company had come to our home to meet my dad. They wanted my dad to be a part of independent scientific advisory panel. The reason: There was immense political pressure on this unit to shutdown, apparently because this unit had been blamed for not enough rains in this region. Logic was because of the windmills the wind was diverted, and somehow the rain clouds didn’t come there! Now now! The committee did submit a report to state government , saying there was no such evidence. To top it all, it rained heavily that year in this particular region. Guess, even rain god don’t like bad science
In India, I don’t see concerted efforts to push for generation of renewable energy: particularly solar and wind which is available in abundance or even the biogas, but poorly utilized. There are ‘here – and – there’ efforts. We do have a ministry of new and renewable energy : http://www.mnre.gov.in/, but I hear more of Petroleum ministry that this more important ministry. I strongly feel that ministry of renewable energy should put in concrete efforts for large scale adoption of renewable energy, which need not be subsidised. A policy like compulsory solar power for all new buildings and existing housing complexes may be a good starting point. 

I strongly feel that Nuclear or thermal energy is just not suitable for our sustainable future. What do you think?

PS. 1) All views expressed here are solely mine. They have nothing to do with my current employer or any organization that I represent.
2) I had though of writing this up from long time. But recent statement on solar power and 'its everyone's choice' by German Chancellor, Angela Merkel, during her recent India visit further prompted me to write this up.

From Symbian to Android

About few months ago I switched from using Symbian to Android. The transition is still in progress, but I can say this much: probably I would never go back to Symbian. And well, Symbian is more or less dead with the Nokia announcement that they would rather be shifting to Windows Phone OS.

As much as this is about moving from one mobile OS to another, it is also about moving from one handset to another. The Symbian phone is a Nokia E51 while the Android phone is a Samsung Galaxy 5.

These are just notes on what I find good and not so good with the switch.

Daily Usage:
My basic requirement from a Phone call is just that it makes and receives phone calls. For this both Android and Symbian seem just OK.

Another requirement from my phone is calender management. In this case Android just wins, mostly because of its seamless integration with Google calender.

Finally battery life: My Symbian phone used to last much longer on a charge than my Android phone. Generally, I feel Symbian is still better in power management. But this may also vary depending on the phone model.

Always Connected:

With Symbian phone I never felt like being always connected. Simply because there were no great apps that needed online update, like no good chat clients.

System stability:

As far as I can say, Android on my phone has not really be stable. With Symbian, my phone worked with a reboot for months ... this has reduced to a few days! This in no way is an advancement. I should really checkout iOS and Windows Phone OS stability to see how other modern OSes fair.

Usability:

I generally find Android to be much more easy to use and navigate than Symbian. Making phone calls, browsing Internet, accessing e-mail, playing music, watching YouTube is all much more enjoying and smoother experience with the Android interface than it has ever been with any version of Symbian.

Did I program?

Not much. Basically, didn't get much time to play around with Android programming as yet. But I did easily manage to port some of the core parts of MeTA Studio project. This initial port is available at: http://cid-76d41f4618b0b6af.office.live.com/self.aspx/metastudio/MeTADroid.tar.gz
May be somewhere down the lane there will be UI Android port for MeTA Studio.

That said, programming on Android is definitely much more enjoying than Symbian C++. I am not sure about the new Qt support for Symbian. But then I always find it enjoying to program in Java than with C++ (i dislike hunting for segfaults in someone elses code ;)).

Push pedal..

An Indian express article (http://www.indianexpress.com/news/pushed-to-pedal-post-fuel-price-hike-a-spurt-in-cycle-sales/796946/) is indicating that there is increase in Push Bike sales in Pune with the recent increase in fuel prices. To me it is just a blip in the timeline. Not sustainable. Unless of course there is substantial up gradation of cycle tracks in Pune.

At least for now I would not actually recommend long distance commuting on a Push Bike in Pune … from my own experience.

Just drawing…

Normal generation for surfaces in MeTA Studio..

For some reason, the earlier builds of MeTA Studio did not have normal generation for surface rendering. This resulted in poor but fast surface rendering. I have just modified this to always generate normals for surfaces starting from r468 revision in the SVN. So now you need not tinker to get a smooth surface.

Remembering Dr. V. Sundararajan, sir

When I called him up yesterday morning on his cell, to invite him for some function, that soothing voice on the other end was no longer there. He had left all of us the very same morning. I am at loss of words.

To me Dr. Sundararajan was an 'un-official' mentor during my Ph.D. program. Although I had quite less academic overlap with him, he always used to help out in umpteen ways, particularly when it came to using the C-DAC facilities for our research work.
He almost always used to attend my Ph.D. progress presentations and had ready useful comments at the end of the session. After I had completed my thesis, he had asked me to give a presentation at C-DAC, which unfortunately never happened as I almost immediately went to ANU for post-doc.

Even when Dr. Sundararajan went into higher administrative roles at C-DAC he was always accessible to everyone who had an interest in doing interesting work with HPC and evolutionary algorithms. He also taught courses at my alma mater, and would indeed be a great loss as a teacher.

R.I.P. may god give his family strength and courage for the life ahead.

MeTA Studio binary release

Another, major bug fix release of MeTA Studio (2.0.17042011) is available for download. The list of issues addressed, and not yet fixed can be seen here.

New additions and fixes in this release:

- apps folder: simple examples of writing new applications on MeTA Studio platform
- MO plot (a new widget is supplied for the same)
- Support for NetCDF and HDF libraries
- problems with i/o redirection for scripts running with in MeTA Studio GUI environment

Still in works:

- Rotation of individual molecules in the editor (partially working)
- Interpolated normals for on-screen iso-surfaces

Configure WiFi usb adapter as a WiFi hotspot (Windows 7)

Had allocated this long weekend for fixing Issue 46 in MeTA Studio, but ended with lot other bugs that had to be fixed. Eventually, I am only half way through fixing the real issue. With all this I got painfully bored :( So decided to do some changes in my setup.

I have a mobile data plan currently on my mobile for which pay about 100 INR per-month for 2GB, which are hardly used up. In fact, I find that I really do not require that. Also, after seeing the new 3G data plan rates (http://airtel.in/Airtel-3G/tariffs.html), I though it was high time I just dump the subscription. So now my phone is quite simply put, un-plugged, with only voice and sms, and pay as you use data.

Now when I am at home, on most of the weekdays, my parents watch TV. And my main desktop as well as the TV is same unit. So, if I need to work (with internet connection), I simply used to setup an ad-hoc connection using a WiFi usb adapter connected to my desktop. The sad part was that my Android 2.1 phone never saw this ad-hoc network. I could simply purchase a Wifi router, but just can't convince myself to get a new device.

After a bit of googling, I came across this very useful article: http://www.blogsdna.com/5506/how-to-setup-virtual-wifi-in-windows-7-without-any-extra-software.htm

Essentially, the whole thing boils down to two commands:

netsh wlan set hostednetwork mode=allow ssid=mywifi key=mypassword
netsh wlan start hostednetwork

and also enabling Internet connection sharing in Windows 7.

Next up, I wanted this to startup automatically when my main desktop booted. I did this by writing the above commands in a batch file and using NSSM - the Non-Sucking Service Manager (http://iain.cx/src/nssm/) to create a Windows auto start service.

Cool afternoon, now back to fixing Issue #46 ;)

Ban! Pretty much an un-Gandhian way

It seems here in India, there is a fad, to BAN.

Recently, many state governments banned a book on Gandhi, proclaiming that it has insulting statements on the father of our nation. The Law ministry is even considering changing the law to make such statements a crime! The bureaucrats and politicians who even think of doing this of course don't seem to have an iota of idea what Gandhiji represented.

When I started writing this post, I thought how can anyone even think of such absurd idea a 'ban'. Let's be quite clear: we are a democratic and intellectual society where the word 'ban' and an associated 'authoritarian' behavior of the people in power has absolutely no place. You can never enforce respect towards a person or a nation or a place, it has to come from within. That is but human mind.

I was quite relived to see that I was not alone to have similar feelings, descendants of Gandhi seem to have much more sense towards what to say, than our learned bureaucrats and government.

My only thoughts at the end of this post: I wish, if the respected law ministry were as sensitive to more real issues like that of Dr. Binayak Sen (http://www.binayaksen.net/) than going down the un-Gandhian way of protecting Gandhiji himself, Bharat would be a far better place to live in!

Note: Any view expressed here are solely my personal views, and have nothing to do with my employer or organizations I am associated with.

HCL Launches ME Tablets

Now, I am not a tablet fan as such, though I do own and heavily use a tablet PC.

But as a mass consumer product, I think, this has a large potential if the price points are right, especially in a market like India, where PC penetration is not that great.

So, I was quite excited almost an year ago when Notion Ink announced their plans to come up with a tablet. And actually delivered couple of months back (http://notionink.wordpress.com/). However, I still don't have a first hand look and feel of it (missed the first pre-order and then the second preorder invite :( ..)

While Notion Ink is quite innovative in what it has designed and continue to do so with their quite open development process, there are other Indian companies that might just get the critical mass in selling tablets, without innovating a lot. Or to say it in other words, provide what a user needs, not necessarily always the "wow" factor.

This is where the news of HCL ME Tablets comes in. And it is not just news, but you can buy the real product online (http://www.hclstore.in/ME_tablet). Of course, this one is not as good in "specs" department as the Notion Ink. But do users really care about specs of a tablet? Steve Jobs wants all of us to believe this, rt? Quite frankly, I think that is true. I don't really care about specs of these devices, if I get a smooth experience and get my job done.

HCL has a history of disrupting the Indian market with low price with pretty high quality products. In fact, my first laptop (or netbook) was from HCL. It was simply the best piece of portable hardware that I could afford back then. And this time around HCL has again come up with a similar strategy. The only difference this time is that in coming months there will be many other players in this space, particularly big Indian mobile manufactures like Micromax.

With all this, I am a bit concerned that NotionInk would actually become a small player, unless they do something radical. Just hope that Rohan is thinking along these lines, or at least reading this post of mine ;-)

Adventurous flight

The funny game

speech, html5 and chrome

Now this is cool (http://chrome.blogspot.com/2011/03/talking-to-your-computer-with-html5.html). Putting speech in browser has just reduced dependence on the PC OSes a bit more. What would be more cool is if it could support more than just English.

So now, the ChromeOS is looking more and more complete. Now when is handwriting recognition coming into picture ;)

A public school in Kanpur

New MeTA Studio binary release

This is essentially a stop gap release that fixes a number of bugs, but also adds a major new feature: MeTA Studio Archive format (.mar). The .mar format allows you to package an application that uses the MeTA Studio platform. This version also includes inbuilt support for Jython, though this is not yet integrated into the UI.

Head to http://code.google.com/p/metastudio/ to get the latest binary package. The current version number is 2.0.19022011 and is also the first binary release in 2011.

Adam, I missed it again! :(

Android finally has official blogger client

So Google has finally released a blogger client for Android. I have been waiting for this for quite some time and was pleasantly surprised to see that in market.

The other pleasant stuff was that Google Goggles is now available for my phone which is pretty cool. Now the only stuff missing is proper offline Google docs support.

Now hopefully I get some time to develop on Android :-)

Is Bing really recycling Google?

Well thats what Google wants you to believe and Microsoft doesn't. I am not a search engine expert, but I just don't believe what others have to say. So I randomly generated a "synthetic search" query: "abduljkastifu" and put it in Google and Bing with the following results.





Which obviously are different. After this, I pretty much stopped reading (http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-google-search.html). The said "recycling" only happens if you seem to be using some settings of IE8 / Bing toolbar (does someone actually use a toolbar?).

But the media putting it simply as Bing copying Google results is what I can classify as being bad Journalism.

Forgotten art: Microsoft Origami for Tablet PCs

Lately have been using my Asus T91 MT a bit too frequently, primary to write documents and occasionally to browse the web. Though I  constantly use the tablet, I hardly thought of two things: faster way to login my machine and better touch friendly window title bar and scroll bars. The rest of the elements in Windows 7 appear to be quite touch friendly to me, but not the above mentioned ones.

During the previous iteration of Windows, Microsoft had released a set of tools called Origami. These tools were specifically meant for touch based UMPC tablet PCs. Two of the tools offered by Microsoft Origami: picture password and touch settings have made a pleasant difference to how I use my tablet PC. You can download Microsoft Origami from here. But note that you may not be able to use other Origami tools on Windows 7 except the ones that I have mentioned. With Asus releasing the "most powerful TabletPC on earth" , probably its time to revive origami for Windows7.

New MeTA Studio icon

Its been long time since there was a public update for MeTA Studio, partly because there is been lot of behind the scene re-architecting going on. And partly because I have been caught up with other stuff.

Even as a number of features were added for MeTA Studio, its icon had never clanged to reflect the new changes. The current icon is a simple rendering of a macro molecule, with the original intent of MeTA Studio being ease of handling large molecules:

Starting with today, the SVN has a new icon originally designed by APR.

The main site also hosts a logo as shown above. The above icon is basically designed with the four core foundations of MeTA Studio in mind: Organize (molecules) , Visualize, Code and Collaborate.

Look back at this space as we prepare to release updates to MeTA Studio in coming weeks.

Motorola Atrix: super phone with a promise

In a recent post, I had argued for a super phone Software interface that:
"Ideally I would like my smartphone OS to automatically scale its user interface form small portable screen to a large desktop scale screen, handle touch, pen and pointing as well as keyboard, seamlessly. That will make computing on the go painless and computing on desk seamless. You will always need only one device: your personal super phone."

With the release of Atrix, Motorola seems to be heading in exactly this direction. Probably a way to acheive this on a phone is via virtualization. That, I guess is how   Motorola is implementing it. With the ever improving ARM SoC capabilities and recent announcement of Windows for ARM, we are soon approaching:

"Three screens, One brain and the Cloud"

mantra. One who gets this right will define the next revolution in computing technologies.

Notion Ink: Good startings

Almost an year ago I started following Notion Ink (http://tovganesh.blogspot.com/2010/01/notion-ink-another-attempt-at-making.html). One thing I was pretty critical about the device then was the software stack, just what sets them apart from other android tablet efforts?

Now with pretty positive reviews from Engadget (Joanna Stern), Liliputing (Brad Linder) at CES'11, I must say that Rohan Shravan and team have pulled up quite well. Good beginnings Rohan et al., though there is still lots to be done ;)

Mypaint: An open source painting application

For some time now I have been searching for a good, responsive, free and professional digital painting tool for my Asus T91 and the bamboo tablet (pc). Today I discovered Mypaint (http://mypaint.intilinux.com/) which is an excellent tool for professional digital painting. Best of all it is open source and simply works. The program can also be configured for pressure sensitivity (provided the hardware supports this) and thus provides most natural painting experience. Highly recommended even if painting is just a hobby for you (like me ;-))