I am happy to announce the launch of FlipThroughMovies.com. This is the culmination of the hack that we worked on during Yahoo OpenHack. You can use this site to navigate through movies available at BigFlix, the popular movie rental site in India.

The site uses publically available data from BigFlix and presents it in very interesting manner - check out the video to get a feel of the site's capabilities. 

*Please view in HD and in FullScreen mode* 

I would like to acknowledge the contribution of the following three individuals in making this site a success.

• Ann Catherine Jose


• Manojayan Vembakkam


• Ram Adhikari


• Yours truly :)

Some technical highlights that might interest you:

• We used Microsoft Silverlight Pivot control to visualize the data
• The site is powered by about 2 GB of data. We are using Amazon S3 to store this data
• To enable efficient delivery, we are using Amazon CloudFront for CDN 'edge caching'.

I will write a different post explaining other difficulties that we faced on getting this site up and running.

I have decided to attend this years Yahoo Open Hack Day - 2009.

More details about the event [http://www.hackday.org/]

• where ... : taj residency, bangalore
• when .... : 14 - 15 february 2009

I am looking for a hacker buddy - so if someone is interested in forming a team, let me know.

I have been using a trial version of ReSharper for my DotSVN project and I should say that I just got hooked to it. I would not have made this much progress without it and that leaves me wondering what I would do once the trial expires!!

A tool with this ease of use is needed to improve the design and code without any hesitation to refactor.
ReSharper's code navigation capabilities are also extra ordinary. You can read more about it here.

I have also used dotTrace to fix a nasty performance issue.

Overall I am very satisfied with the product.

I would like to write a series of posts describing the various performance issues I faced while implementing DotSVN.

In this post I discuss the performance penalty of using DateTime.Parse().

During my testing, I found that DotSVN was running quite slow. I used a trial version of 'JetBrains dotTrace 3.0' to analyze the problem. The following is a screen-shot of the dotTrace session.


As you can see, 26% of the time is spend on the method call 'System.DateTime.Parse()'. This was unacceptable. After some investigation I found that the performance of the DateTime Parse method can be improved if we give some clue on formatting of the date string. This can be achieved using the ParseExact() method.

note: I have updated the code to re-use the CultureInfo class.
AlexKucherenko - Thanks for the comment.

private static readonly CultureInfo en_us_Culture = new CultureInfo("en-US");

// Parse in the format [2007-09-06T10:20:26.689093Z]
private static readonly string dateTimeFormat = "yyyy-MM-ddTHH:mm:ss.FFFFFFFZ";

public static DateTime parseDate(String dateString)
{
	DateTime parsedDate;

	bool parseResult = DateTime.TryParseExact(dateString, dateTimeFormat, 
							en_us_Culture, 
							DateTimeStyles.AdjustToUniversal, out parsedDate);
	if(!parseResult)
	{
		SVNErrorMessage err = SVNErrorMessage.create(SVNErrorCode.BAD_DATE);
		SVNErrorManager.error(err);    
	}
	return parsedDate;
}

The performance gain is quite obvious with the following figure, which shows the dotTrace session after applying the above fix.


As you can see, DateTime.Parse() is no longer a hot Spot. This also shows the power of dotTrace, and how it helped to quickly narrow down the issue.

I have been using Firebug Firefox plugin. This is a really useful tool, and I would definitely recommend this for anyone who is interested in Web development. If you are new to this tool, I suggest that you go through this article, as a good introduction.

I have captured a few scenarios where this proved useful to me.

JavaScript debugging
Javascript debugging is a really compelling feature of Firebug.



As you can see, you can easily put break-point and step through .js loaded in the browser.

Network Monitoring

FireBug can present a time-line of the various resources downloaded to render a given page. This is very useful for understanding performance issues with a page.

XML HTTP Request monitoring (XHR)

With this feature, we can track the actual request and response between the browser and server.
I used this feature to track the JSON response from the DotSVN server.

Dell announced the availability of the new Inspiron 1720 line in India. The new Inspiron is built on Intel “Santa Rosa” notebook platform and is a budget-friendly, desktop-replacement. I was looking at upgrading my laptop and thought this is a nice time to do it.

The specification that I was looking at is as follows: Intel® Core™ 2 Duo T7300 (2.0GHz/800Mhz FSB/4MB cache) Windows Vista(TM) Business Glossy widescreen 17.0 inch display (1920 x 1200)  => I really like this :-) 256MB NVIDIA® GeForce® Go 8600M GT 2GB Shared Dual Channel DDR2 at 667MHz  => The platform supports up to 800Mhz FSB, it is a shame the memory is this slow ;) 160GB SATA Hard Drive (5400RPM)  => There is provision for an additional SATA HDD, but no RAID :( CD / DVD writer (DVD+/-RW Drive) Integrated 2.0 Megapixel Webcam

I have been working (along with my sister and cousin) on an XMPP based IM client. This is fully implemented in DotNet. The current version (1.0.1) can connect to Google Talk. The project is hosted here. 

You can get the latest code from the following SVN url:
http://chatapp.googlecode.com/svn/trunk/

Here are some screen shots of the application.

 

And the Chat Window

The 'About Box' describes the different components that we are using.

 I would appreciate your valuable comments - please post them here.

Microsoft announced the availability of SilverLight 1.1.

You can get started by visiting here.

Microsoft Silverlight is a cross-browser, cross-platform plugin for delivering the next generation of .NET based media experiences and rich interactive applications for the Web. Silverlight offers a flexible programming model that supports JavaScript, .NET, and other languages.

This is really interesting. A 4 MB download that contains a Cross-Platform CLR and which supports a good subset of the BCL - including support for .Net 3.5 stuff like LINQ.

Today, I got introduced to a tool called Hamachi. Simply put Hamachi is..

... Hamachi is a zero-configuration virtual private networking (VPN) application.
In other words Hamachi is a program that allows you to arrange multiple computers into their own secure network just as if they were connected by a physical network cable.
Hamachi is fast, secure and simple. Its core version is also free.

...

Hamachi gives you LAN over the Internet.

Virtually any application that works over local/home networks can also be used over Hamachi networks. Think - Windows File Sharing, iTunes, Remote Desktop, Remote Assistance or even gaming - all fully encrypted, authenticated and peer-to-peer.

...

Hamachi is a zero-configuration virtual private networking application with an open security architecture and NAT-to-NAT traversal capabilities.

In short, I was able to use Remote Desktop to connect to the PC of my sister and the performance was good.

Once you login, you will get a unique IP (usually starting with 5.). You can create/join networks. Once you are connected to a network, others already in that network can connect to you using your IP. 

I came across some articles that describe SVK as a better version control system, than say SVN.

svk is a decentralized version control system built with the robust Subversion filesystem. It supports repository mirroring, disconnected operation, history-sensitive merging, and integrates with other version control systems, as well as popular visual merge tools.

I already have an SVN server running at my home PC, and I am quite happy with the 'Tortoise SVN' client. I came across SVK, when I was researching the possibility of maintaining an SVN mirror repository - especially useful if your SVN repository is hosted on a remote server.

I will be documenting my experience with this, but if you have any comments, please do share... 

I started using HiveMinder from Best Practical.

Hiveminder is a web-based tool to help you get (and stay) organized. Hiveminder helps you keep track of what you need to do, assign tasks to anyone with an email address and share tasks with anyone you collaborate with.

I liked the fact that you can import tasks into the system by uploading a text file.

Ken Schaefer has started a series on the IIS and Kerberos. You can read the first part here.

Authentication is the process of proving your identity to a remote system. Your identity is who you are, and authentication is the process of proving that. In many systems your identity is your username, and you use a secret shared between you and the remote system (a password) to prove that your identity.

This is a continuation of my previous post on OpenID.

Today I will explain, how you can personalize your OpenID URL when using the VeriSign Personal Identity provider (PIP) system.

Step 1: You should create a OpenID account with your Identity provider. In my case, my OpenID is http://georgeck.pip.verisignlabs.com. Now, that doesn't sound like a name I would want to remember or associate as my OpenID. Another reason why I wouldn't want to use the default ID created by my service provider is that, I don't want to be tied down to a specific vendor. What if my identity provider shuts down!

Step 2: Create a unique URL which you can control and setup a redirect to the Identity Provider. Again, taking my example - http://george.chiramattel.com is my site and I have full control over it. I edit the default page there and add the following meta info into the HTML header. (Remember the following example is for me and VeriSign PIP).

<html>
<head>
<link href="http://georgeck.pip.verisignlabs.com/server" rel="openid.server" />
<link href="http://georgeck.pip.verisignlabs.com/" rel="openid.delegate" />
<head/>

.. followed by the rest of the HTML document ..

This instructs the requesting site to redirect to my actual Identity provider. At a later point in time, if I wish to change my Identity provider, All I need to do is modify the above snippet and point it to my new provider.

First, let me define Identity 2.0. Wikipedia defines Identity 2.0 as....

Identity 2.0, also called digital identity, is the anticipated revolution of identity verification on the Internet using the OpenID standard. Identity 2.0 stems from the Web 2.0 theory of the world wide web transition. It's emphasis is a simple and open method of identity transactions similar to those in the physical world, such as driver's license.

OpenID is not the only player in this field. Other solutions include:

• InfoCard from Microsoft
• Liberty Alliance
• i-names  and the list is growing...

I also found a solution from whobar that

allows users to login to a website using InfoCard, OpenID or i-names and makes it all transparent to the web application

in which case the login screen would look something like this.

So, what does all this mean to the end user?

To understand this better, let me take you through the experience of authenticating myself to an OpenID enabled site. In this example I have used the free OpenID service provided by VeriSign called Personal Identity provider (PIP).

Step 1: I want to login to an OpenID enabled site. I am using Opinity as an example. To login, I am presented with a text box to enter my unique identity URL. Notice the OpenID logo in the text box.

Step 2: Once I enter my unique ID, (in this case george.chiramattel.com), I am redirected to my Identity provider, which is VeriSign. I login to VeriSign and I am presented with the following screen from VeriSign. Here I can authorize VeriSign to 'prove' my identity to the referring site - Opinity. As you can see, I can set a timelimit to my authorization. I can also choose to share some personal profile information that I have previously stored with VeriSign with Opinity - but in this case Opinity is not expecting any such information.

Notice that VeriSign mentions my name as http://georgeck.pip.verisignlabs.com. I will explain how I could configure this in a later blog. Once I confirm my authorization, I am redirected back to the referring site (Opinity) and (ideally) I should be logged in.

How is this different from a normal web-signin?

To start with, I don't have to maintain different passwords for different sites. My identity is my unique URL and I can present it to any site to let them know who I am. Now to prove it is really me who is presenting this URL, I use the service of the OpenID server, who ascertains my claim on my behalf.

I will explain details on how I configured my OpenID in a later blog.

What is Dapper? Well, according to the creators, dapper is...

... a service that allows you to extract and use information from any website on the Internet. For those familiar with web services, you can think of Dapper as an API maker. For the rest of you, Dapper allows you to build web applications and mashups using data from any website without any programming.

This means that you can treat the web like a database - well, almost. You can also call it a glorified screen-scrapper' with an Ajax frontend and a service to back it up.

A long time ago, I was inspired by an article with a similar theme. The title of the article was '.NET Web Data ToolKit'. You can find it here.

If you compare the two, with Dapper, we get the data published in standard, easily consumable formats like XML, RSS, JSON etc. With 'web data toolkit' (WDT), it is more programmatic and the results are returned as ADO.Net data sets. The author Tony Loton describes WDT as...

...a toolkit that allows you to SELECT nuggets of information FROM live web pages as though selecting from tables in a relational databas, using industry-standard SQL. Imagine if you could capture the results into an ADO.NET DataTable, allowing you to treat web data like any other data source.

Though this is a powerful idea, it has some drawbacks. To start with, we are dealing with some-one-else's data - it is after all screen scrapping! If you are OK with it and wants to take it to the next level - say, take any set of web site(s) and then mash them up to create totally new web services, then you stumble upon the next drawback. Which is the lack of a workflow.

In most non-trivial usecases the website exposes the data by taking the user through a sequence of steps. An example step could include a login, followed by a click on a link (say my account name) which than takes me to to my data (say my account details). So to get to the 'real' data, the user have to follow multiple steps and certain steps could be involve HTTP 'POST' and not get just simple "GET's. So the service should have support for executing a workflow of steps. On each step, we should be able to control the actual HTTP data being send to the server and also be able to specify alternate control path if things go wrong. So the support for a WorkFlow engine is critical.

The next problem with Dapper would be privacy. I cannot create a web service with Dapper to get to my personal data (say my bank details) as I would have to share my login credentials with the site. So what we really need is a toolkit that I can embed in my application - similar to Web Data ToolKit.

I should admit that I was not able to spend adequate time on FolkMind. Now that I am back from my vacation, I am committing myself again to this. And this time I am hoping that I can maintain the momentum.

Google started an initiative called 'Google Apps for Domains'. According to Google, this service will..

..offer private-labeled email, IM and calendar tools to all of your users for free*, so they can share ideas and get things done more effectively. You can design and publish your organization's website, too. It's all hosted by Google, so there's no hardware or software for you to install or maintain.

One of the first steps that I did is to migrate the domain folk-mind.org to the above Google service. I should say that my migration experience was quite painless and was able to finish the whole activity in less than 2 hours (including adding the default home page).

My only crib is that, currently there is no option to change the 'A record' in the DNS. We can only modify the CNAME record. This results in folk-mind.org and www.folk-mind.org (this time with the www) returning two different pages. My brother helped me resolve this by using a web forward to folk-mind.org.

I am writing this post using Windows Live Writer (Beta). Currently the beta is available for download here.

I am pleasantly surprised with its capabilities. To start with, it can connect to blog services other than Windows Live Spaces. And once it is configured, it downloads even the visual theme from the blog site. So while you are editing the blog, the visual theme is presented.

the tool also exposes a plugin API (currently available here).

Quite a few plugins are already available. One of which (flickr4Writer) allows you to insert images from Flicker (and this is the plugin I used to insert the above picture).

In the article titled 'The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software', Herb Sutter explains the importance of improving our skills on writing concurrent programs - ie if we need to gain performance gains form all the multi-core CPUs that are coming out lately.

In this light, I have collected a series of articles that discuss this matter in detail.

• An Introduction to Programming with C# Threads
• Threading in C#
• CLR Inside out: Using concurrency for scalability [MSDM Mag Sep 2006]
• What Every Dev Must Know About Multithreaded Apps [MSDN Mag Aug 2005]
• Advanced Techniques To Avoid And Detect Deadlocks In .NET Apps [MSDN Mag April 2006]
• WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred
  

Hope this helps.

technorati tags:Threading, Concurrency, CSharp

This is to list the set of web 2.0 tools that I am using to plan and track my vacation.

Google spreadsheets:

We are using Google Spreadsheets to consolidate all the data that we are getting from the web.

This is really useful when multiple people are simultaneously gathering data (my brother and sister-in-law who are currently in Belgium are also actively involved). With this tool, I can share my spreadsheet with multiple people and all of us can simultaneously edit the contents. This also helps in keeping the data always in sync.

Great tool!

Yahoo Trip planner:

Yahoo trip planner helps in trashing out the next level of details in the trip schedule.

The process starts by specifying the start and end date of the trip. Once this is doe, we can start to add meat to the schedule.

Yahoo also facilitates this process:

• It lets you check what others have done on similar vacations
• Helps you decide on what to do at different locations, by giving a list of things to choose from (with user reviews/ratings).
• Also integrates with Flickr. So if you have uploaded some photos, this tool helps you associate different photo with different locations.
• Helps in creating a trip journal.

I have just started my planning here. Again, this tool helps in keeping the plans made by all of us in sync.

Google calendar:

Though Yahoo trip will help you create an agenda for the vacation, if we need more fine grained detail, we need a real calender. That is where Google calendar comes in.

Google also lets you publish this calendar where ever you want. And of course, multiple people can simultaneously add to the calender.

I have published our calender at my site here.

technorati tags:Web 2.0, Yahoo Trip Planner, Google Calendar, Google SpreadSheets, Europe Vacation

I wanted to try out Google Maps API. As a sample I tried to embed a Google Map showing the location of my apartment.
Please find the sample here.

If you zoom out three levels you can see the Bangalore Airport.

After installing Google Desktop 2.0, I have observed that my Lotus Notes tends to crash whenever I browse web pages from within Notes.

An easy workaround that I found is to configure Notes to use an external browser.

technorati tags:Google Desktop 2.0, lotus Notes

I guess this is self explanatory...

technorati tags:Vista

I managed to install 'Windows Vista Ultimate' Beta 2 (Build 5384) on a Dell Precision work station 360. The machine was running a 3.2 G Hz Pentium 4 and had 1GB main memory and 80GB SCSI disk. It also had a NVIDIA Quadro Fx 500 display card with 128 MB graphics memory.

Pre Windows Vista, the above configuration can be considered 'good-enough' for normal usage. But I was surprised to find that the Vista gave it an overall System Performance Rating of just 2!

With this rating, Vista will not enable the 3D desktop features. I guess, 128 MB graphics memory is too little for Vista!

technorati tags:Vista, System Performance Rating

I have subscribed to Safai Books Online. This is an electronic reference library (with over over 3,700+ books) targeted for programmers and IT professionals. It allows you to:

• Search across the full text of thousands of the best technical books.
• Browse books by category to research any technical topic.
• Read books cover to cover online.
• Or even download whole chapters for printing and viewing offline.

The publishers include

• O'Reilly
• Addison Wesley
• Sams
• Prentice Hall
• Que
• Cisco Press
• Microsoft Press
• Peachpit Press
• New Riders Publishing
• Alpha Books
• Course Technology
• IBM Press
• Macromedia
• Adobe Press
• Syngress
• and others....

If you are interested, you can subscribe for a 14 day trial here.

Happy online reading!!

technorati tags:ebooks, Online reference

Found this link containing some very interesting papers on Computer Science.
Definitely a 'must-read'!

I see that most of the samples written for Flash Player 9.0 beta, breaks with this new release.
We might have to recompile the code with the released SDK.

Flex 2 is now available for immediate purchase from Adobe.com. With the new shipping status of Flex 2, the Flex product line (including Flash Player 9 and ColdFusion/Flex Connectivity) has been retired from Adobe Labs. You can download the final, shipping versions from Adobe.com.

Get more information on these shipping technologies >
Download a free trial of Flex Builder 2 and get the free Flex 2 SDK >
Download Flash Player 9 >

Adobe Labs: Flex 2 Is Now Shipping

I downloaded Flok Beta 1 (v 0.7) today. I am really happy with the quality of this browser.
The following is how they describe the functionality of Flok.

Flock is a free web browser that makes it easier than ever to share photos, stay up-to-date with news from your favorite sites, and search the Web.

This blog entry is made with Flock!

technorati tags:Technology

Blogged with Flock

I have been discussing the idea behind FolkMind with one of my friend and the conversation went something like this:

Folk: So George, I have heard that you have coined this new term ‘FolkMind’. Tell me more about the idea behind it. I didn’t get time to go through the long blog that you have written on this.

George: It is about a new way of retrieving information from the Internet, using the collective intelligence of…

Folk: Oh, then that’s what it is, another search engine! Haven’t you heard about Google!!

George: Well not exactly. A good search engine will help you find information, but it doesn’t help you in understanding it. But FolkMind will!

Folk: What do you mean by ‘make you understand’?

George: Let’s take an example. Say you are searching for something like ‘Web 2.0’. Google returns you more than 15 million records.

Folk: Exactly you got the information that you wanted. It was that simple.

George: But that is too much information and like I mentioned in another blog, too much information is no information.

Folk: But the links that you got from Google are sorted based on the relevance. You know they use this algorithm called ‘PageRank’ which evaluates what the Internet community has to say about the subject being searched for.

George: To handle as huge an information source as the Internet, you require a totally decentralized community driven approach like PageRank. That’s obvious! But I still have to go through multiple pages to find out and understand what I was looking for.

Folk: Do you expect to find pages that exclusively contain what you are looking for?

George: Well, I don’t expect that! When you ask a question to an Oracle, you expect her to give a single answer instead of 50 different instructions, which you have to club together. Similarly, wouldn’t it be nice if the search engine could return you a page that is generated for me based on my query?

Folk: You mean an auto-generated page? Come on George, you must be kidding. I would prefer to read through 10 man-made pages to get what I want, than to go through all the junk returned by a bot.

George: Well, have you seen a newspaper that is made completely using an algorithm? The newspaper is called Google News!

Folk: Yes. I know about it. But news is different you know..

George: How come? As you would already know, Google News allows you to customize the news based on your preferences. So, in a way, it is also a search that you perform and the algorithm generates a page for you.

Folk: Hey, but Google News also just lists a set of links; you still have to navigate to the source page to make sense out of the information

George: That could be because Google doesn’t own the news and hence expects you to visit the original page. Well, coming back to the discussion on search, the result of a query need not be text that reads like a story; instead it could be a diagram that represents the essence of it. That way any bot that generates the result wouldn’t have to worry too much about structural stuff like grammar and readability. Think of the benefits – a picture is better for easy and quick comprehension as you would be using both halves of the brain when you ‘read’ a picture. Also, if you bookmark the picture instead of the text article, you will be able to recollect the matter by just glancing through it instead of reading the whole stuff again.

Consider the following facts:
 * 60% of what you read are structure words and not content.
 * 80% of the stuff that you read is forgotten within 24 hours.

Folk: What are you trying to say - that I am wasting my time in reading?

George: If your intension is to comprehend as much stuff in the shortest possible time, then reading verbose text is not the way for doing it. Say for example, suppose you were asked to review 50 code files. Would you start by reading from File1, File2 etc. or would you look for a UML diagram and then drill down for more details?

Folk: I would prefer UML diagrams and I agree that pictures are better than normal text for comprehension. But how can you compare structured data like code with text that you find on the Internet: You cannot create ‘UML’ like diagrams out of free text.

George: Yes. You are right. But if you do not want very precise data, why not represent the text as a MindMap. If you need more detail on any specific data you can always look at the original text by clicking on the appropriate section on the MindMap.

Folk: Hmm.. Sounds interesting. So, I do a Search and the result that I get is a MindMap that is generated by combining the related information from different places on the Web. What is returned is knowledge instead of links to thousands of sites where I can find the data.

George: Also the knowledge is presented in an easy to comprehend manner – as a MindMap.

Folk: But I know of a few other sites that do similar stuff, like Grokker and …what is the other name ...yes, Kartoo. How is FolkMind different?

George: Consider this. When you do a Google Search for say “Microsoft”, you get a few hundred million links. Anything beyond the first few pages are not useful for me. The tools that you mentioned above are designed to just solve this very problem. The value addition provided by these tools is that they can better represent the search results graphically. Grokker does a good job of categorizing the results. Kartoo goes one step ahead by also finding out common links between the results. So they are search enhancement tools. But as you see, FolkMind is more focused on retrieving “knowledge”.

Folk: Can you explain this better?

George: Sure. What you expect out of a search is not a list of probable links or their graphical representation. What you need is the “knowledge” based on your context.

Folk: Context? You mean..

George: When I do a Search, I am trying to extend my current knowledge along a particular axis. If a search engine really knows “me”, then it can return more relevant information. This would want the search engine to be aware of my current knowledge set

Folk: Hmm. I agree. This could be the reason behind the current trend of personalized search, search history and so on… so that the search engine knows my interests. So how does FolkMind satisfy this need?

George: To start with, FolkMind is a digitized version of the knowledge that you have. It helps you in extending you current knowledge by utilizing the Internet. The Internet is the biggest source of information and in a way represents the virtual brain of humanity.

Folk: A digitized version of my knowledge – what do you mean?

George: Let me take bookmarks as an example to explain this. When ever you find something interesting on the web you bookmark it. Right?

Folk: Yes – Obviously I won’t be able to recollect every thing that I have ever read and bookmarks help in recollecting stuff later on.

George: Precisely. Now assume that instead of bookmaking stuff, you use FolkMind to create a MindMap and store that instead of the bookmark. So when you have to recollect stuff, you can glance through the MindMap and be done with it instead of going through the whole text. Also assume that when adding new maps, FolkMind helps you in cross-linking it with previously stored maps. Over time this store will become a huge collection of interconnected concepts.

Folk: I can imagine. In a way this would resemble the human brain itself. Just like in the brain, we have neurons connecting to other neurons; FolkMind would have concepts connected to other concepts. Now I get what you mean by ‘digitized version of ones knowledge’!

George: The value of such a store will increase as the amount of data in it increases.

Folk: But George, that would be a lot of effort to create such a store. Do you think people will be willing to put in that extra effort?

George: Think of it like this. On one hand we have the extra effort to populate the MindMap and on the other hand you have the benefit of recollecting stuff easily. So if you reduce the effort in creating the MindMap store, the benefit could be enough motivation to put in that extra effort.

Folk: Well, you never know. Most people could be like me –Very lazy :-)

George: Ha Ha .. I know. That’s where the community can help. Take the example of Wikipedia. Only a fraction of the people takes the effort to update the articles. But many people use it.

Folk: How do you propose to reduce the effort for creating the MindMaps and that too using the community? I thought you just said that FolkMind would be a local application!

George: Think of FolkMind like Google Desktop. Yes, It runs locally but it can connect to the FolkMind server to get more data. Let me take an example. Assume FolkMind runs as a browser plug-in, always accessible to you. When you browse to a page, FolkMind can do some ‘Natural language processing’ on the text and will try to automatically create a MindMap and this is done in the background. Also the resulting MindMap is cached. The similarity with Google Desktop is that the former tries to index what ever you see; FolkMind tries to create a MindMap out of it. You can imagine that this auto generated MindMap would not be perfect. That’s where the user can help. If he is really interested in a particular article, he can invoke the plug-in and brings it to foreground. This will result in the MindMap graphically displayed over the article as an overlay. Also it presents tools for the user to quickly make corrections to the MindMap, like add, delete or modify nodes or connection. When a user bookmark this article, the MindMap thus generated is also linked with it.

Folk: Ok, now I understand how Natural Language Processing done by FolkMind can help reduce the effort put in by the user in creating MindMaps. How can the community help here.

George: This is the most important aspect of FolkMind. Any MindMap that is generated by the user is also saved on the server in his profile. The advantage of this is that wherever you connect, your MindMaps are always available to you. Also, FolkMind server can also share this data with other users. So the content that you helped create is used to increase the accuracy of MindMaps generated when someone else looks at the same page. Also you benefit from what others have contributed. This collective value addition is key to increasing the accuracy and relevance of the MindMaps generated. As more and more people use it the content becomes accurate and relevant. Over time more pages get stored and cross-linked.

Folk: WOW that’s a very powerful idea. FolkMind server can help create a Concept Map for the whole Internet. This will help change the way we look at the Internet. Instead of considering the Internet as a collection of web pages connected together using hyperlinks, using FolkMind we can visualize the Web as a collection of interlinked concepts. So, instead of navigating through hyperlinks, you can navigate through concepts.

George: This will also change the way you search for stuff. Instead of doing keyword searches you can do concept searches. And since FolkMind already knows about your context or knowledge set based on what you have already stored in your profile, it can retrieve you stuff that is very relevant to you or even suggest new stuff. It is like Amazon suggesting things to you based on what you have looked for before.

Folk: So if my knowledge set already contains stuff like Web 2.0 and AJAX, FolkMind can suggest to me things like Atlas as and when they emerge on the Internet.

George: Precisely. It can help you create new useful connections between existing concepts. In essence it can help you create new ideas!

Folk: Now I am really exited about this. How far have you finished this?

George: FolkMind is still in a very early stage of development. I have created a project on SourceForge. If you are interested you can join in. Also you can help spread the word by digging it here.

The following is an article that I co-authored (along with Brijesh Chenan and Rahul Sawhney) almost 4 years back. We represented Philips Software and presented this for the ‘Road Ahead Contest’ conducted by Microsoft India and got selected as the best paper :-)

But why talk about this now?
Well, I came across this document when I was going through some of my old papers and what struck me was that, most of the stuff that we predicted in this paper have (or is in the process of) become a reality. So please read on…

Introduction
Today we are witnessing another era of explosive growth on the web. The web has come a long way, from static content to dynamic content and now to programmable web services. The Internet has metamorphosed into a pervasive computing domain, providing value-added services. This means that the web is now available in ways never imagined in the past.

The web is now mature enough in terms of availability of services. All the functionality that you would want is probably already there on the web. But the truth is that only a knowledge worker can harness the full power of the programmable web. Further there is no integration of web services into applications that people use daily. It is time to start thinking about bringing this power to the common man.

This paper proposes some enhancements to the existing web services framework for better integration between web services and applications accessing them.

Need for Seamless Integration
While embarking on a web services approach, several important questions come to mind.

As web service providers,
* How can we make it easier for our customers to use our web services?
* How can we increase our client base?
* How can we ensure a viable business model around our web services?

As application developers,
* How can we make our applications more powerful by harnessing the power of web services?
* How can we provide our customers access to web services that are “yet to be developed”?
* How can we ensure that our applications are state of the art and provide the latest features without massive development costs?

The answers to these questions lead us to the next generation of killer applications.
These applications will be able to utilize the zillions of services out there and the ones that are yet to come. Armed with the knowledge of web services, they will be able to proactively help users in getting their work done. As developers we should focus not only on exposing services but also be capable of locating and binding to existing and future services on the net.

To illustrate, let’s look at the following scenario.

Scenario

Suppose you are interested in online trading and would like to check out the top-ten movers. Further, you might also want to transact on them. In today’s world, typically you would use your browser to search the web for online stock trading services. You would then choose one of these services and then navigate to its site. Here, you would click the link for the top-ten movers. In the list returned, you find MSFT [Symbol for Microsoft Corporation stock at NASDAQ] interesting and want to buy 100 shares. To do this, you would click the corresponding link on the site and make the transaction. Now, suppose you want more information about MSFT from some other service, or make the trade elsewhere, you would have to perform the same operations all over again at the second site. You obviously cannot use the data you obtained from the first site to invoke services at the second site.

Let us see how the killer applications of tomorrow alter this scenario. You fire up the next generation Excel and run a web query on the top-ten movers. The web query returns the results from the web service and formats it on screen. You find MSFT interesting, so you right click on the symbol ‘MSFT’. The context menu shows you the option “Web Actions -> Get Detailed Stock Quote…”. When you click this option, it returns the latest stock quote update on MSFT. You then decide to buy 100 shares, so you right click and select “Web Actions à Buy this stock…” on the context menu and go on to complete the transaction (See figure below).

Using Excel, you query your portfolio (probably from another web service) and see your open positions. You see that you are long on ‘MSFT’ and decide to sell 50. You go to the corresponding field and edit it to 50. Excel automatically prompts you “Do you want to sell?” You press “Yes” to complete the transaction.

How to realize this

This can be achieved through a combination of two enhancements to current web services framework.
* Mapping between data and web methods
* Mapping between common action and web service methods

Mapping between data and web methods

The WSDL [Web Service Description Language] file provides the application with an interface description of that web service. Each of these web methods returns XML data. With the current standards, it is not possible for the application to figure out what further can be done on these data. The application will have to rely on custom hard-coded logic to invoke further actions on the data. If we have another XML file that will contain the mapping between the data elements and what actions can be invoked on them, the application can be much more proactive in helping the user get the work done. These actions relate to other web-service methods, which could be provided by either the same or some other web-service provider.

In short we propose to provide a link to an XML file within the WSDL file for a web-service. Bodies such as W3C [World Wide Web Consortium] can standardize the schema for this XML file.

With this in place, the application in the illustrated Scenario will be able to provide context actions like “Get Latest Stock Quote” and “Buy this stock” because it automatically knows what can be done with the data.

We feel that the concept of associating web services and data is very powerful. It opens up new avenues for applications to interact with multiple web services on behalf of the user without him having to write any programming logic.

Mapping between common actions and web service methods

We have seen how the mapping between data and actions will enable the user to invoke services for that data. But the user has to initiate this action, probably through a context menu. We can take this to the next level by making our applications intelligent so as to invoke corresponding services automatically.

In the above Scenario, when the user modified the field containing the number of stocks in his portfolio, Excel understood that the user is doing an ‘Edit’ action on that data item (which was returned by a web service). It is now up to Excel to find out which method corresponds to the ‘Edit’ action for the data item being edited. Once this method is identified, consent from the user is obtained to invoke it on the server to make the transaction.

A standardizing body can define an XML file that lists certain commonly performed actions (e.g. verbs like edit and delete). Next generation applications should be aware of these verbs defined in this XML file and be able to translate user-actions to them. A service provider while publishing the service can also provide a mapping XML file. This file specifies the verbs associated with data that is returned from each web method. In turn, each verb is mapped to a web method that is invoked when the user performs the corresponding action at the application side. The schema for this file should also be standardized.

Conclusion

The Internet will continue to evolve as a source of computational services rather than a repository of content. The web will gradually move towards a state where more and more value-added services will be presented as web services. The killer applications of tomorrow will be the ones that can seamlessly integrate these web services. These applications will replace the browser as the main interaction point to the web and blur the boundary between the local and the remote.

The standards of today focus more on how to locate, identify and bind to services. We should also focus on standards that define consumption of these services. This will allow applications to behave intelligently to identify and invoke services from disparate sources, transparent to the user.

You can also download the PowerPoint that was made for this contest here.

Explosion of content on the Internet

Currently, a lot of information is being published in the Internet as blogs, which are rich in content, frequently published and scattered across numerous sites. MSDN Blogs alone hosts around 2000 blogs. It is very difficult to aggregate this information and get a consolidated view of the same. RSS readers and content aggregators have tried to help us achieve this to some extent, but these tools cannot scale up to handle the voluminous and widespread content in the Internet.

The brain as a better model for describing the Internet

The Internet in its current form represents the collective intelligence of humanity and it has many characteristics that are similar to the human brain. Just like new connections are made between neurons in our brain as we learn new things, as new concepts emerge on the Internet, the links between the documents that describe the concept gets stronger – quite similar to the organic growth that we see in the brain. Also in the brain the connections between the neuron are more important than the individual neuron (though the neuron forms an integral part of the whole, they are far too many to be considered individually). Similarly in the web the concepts that emerge out of connected documents are more interesting than the individual document itself. One might argue that a document(s) authored by a few authoritative person(s) on a subject might be a better representation than the collection of documents made by the whole crowd. Though it might sound counter intuitive, in his work titled 'the wisdom of crowds' James Suriowecki explains that a better representation and a more accurate picture emerges out of the collective decision of the crowd than the work of a selected few. This is also probably one of the factors behind the success of Wikipedia as an accurate encyclopedia.

Currently the most prominent means of getting information from the Internet is through search. This approach is good enough if we already know what we are looking for – or if we know the right question to ask. But if we visit the Internet with the intension of finding something new, a better approach would be to navigate the Internet directory or taxonomy.

Benefit of Folksonomy in favor of Taxonomy

Lets take the previous use-case of browsing through MSDN blogs, looking for something 'new’ and ‘interesting’ – this is a good example of a situation where I wouldn’t know what exactly to search for. Instead of individually going through each blog, I can extract the statistically unique terms to form a taxonomy. My intension in doing this exercise is to distill the contents of more than 2000 blogs into a few words and then pick out (from the resultant set) those terms that I find to be interesting. This can be easily done using TagCloud and the result is as follows.

You can see the original cloud here

Is this auto-generated taxonomy good enough representation of what is published through MSDN blogs? In my opinion-No. This result is purely statistical in nature and I would compare it to the result of Google index with out 'page rank'. It does not take into account the collaborative content selection and filtering that happens usually on the Internet. It is this additional data that makes the data more relevant. A better approximation would be to use the celebrative tagging also referred to as 'Folksonomy'. The term folksonomy is defined in wikipedia as "a neologism for a practice of collaborative categorization using freely chosen keywords. More colloquially, this refers to a group of people cooperating spontaneously to organize information into categories, typically using categories or tags on pages, or semantic links with types that evolve without much central control."

This kind of tagging allows for the kind of multiple, overlapping associations that the brain itself uses, rather than using rigid categories. Such flexibility in using tags is both good and bad. On one hand we have tags like 'blog' and 'blogs' appearing as different tags. On the positive side, a photo of a smiling baby might be tagged 'baby', 'happy' and 'cute'. So in effect folksonomy produces results that more accurately reflect the population's conceptual model of the information.

The need for better tooling

If the brain is a better representative model of how the Internet works then we need different kind of tools to navigate and retrieve information from it. To be able to cope up with the vast amount of information, it should be capable of navigating across concepts instead of across documents. And once we locate our exact match, we should be able to drill down to it. To explain this new UI, lets take the example of Google Earth. To be able to locate a particular spot on earth (which is not previously tagged by Google - hence not available to search), we can take two approaches. The brute force approach would be to hunt through all the locatable points on the surface, until we reach our point. A more efficient strategy would be to zoom out (in other words elevate our self to a higher altitude), where we get an over all picture and then drill down to our point of interest. So to handle more complex problems, we need to create better abstractions. Another benefit of higher abstractions is that, at higher level we can easily spot associations and connections between locations (or concepts) that are hard to find at ground level. In the analogy, just as we are able to navigate across continents, countries and states, we should be able to navigate across concepts that emerge out of the Internet. Another vital feature that is missing in the tools that are currently available is the ability to discover and make connections between concepts. It is this lack of tooling that led me to envision FolkMind.

The vision of FolkMind

To me the new killer app for the Internet should help me in working at any levels of abstraction. The higher the abstraction, the more volume of complexity and data I can handle. Also at any level of abstraction, I should be able to navigate between concepts that are visible at that level and observe new connections that were not apparent to me at a different abstraction. And when I want to dig deeper, it should help me in exploring more on that subject. At the lowest level of abstraction, it would resemble a browser. The mass of the content that is on the Internet will still be on HTML, which is doing a good job of capturing presentation information, and a browser is suitable to view this. In short this application should act as a seamless extension to mind and help me in generating ideas by creating new connection between concepts about which I have little or no previous knowledge by leveraging the collective intelligence of humanity. A 'mind map' would be a good UI (an example of such a UI is shown below) for representing the above-mentioned vision. Wikipedia defines the term 'mind map' as 'a pictorial representation of how a central concept is linked to other concepts and issues'.

As a start we can create a mind map of the existing folksonomy that already exists on the Internet (with data from sites like del.icio.us) and then add new content and nodes to it. In his article entitled "Using Wikipedia and the Yahoo API to give structure to flat lists", Matt Biddulph explains a simple method for automatically converting a set of terms into a connected graph. To me the idea of linking together concepts is quite powerful. Once we reach a critical mass of concepts defined in such a mind map, it can transform itself from a concept management tool to an idea generation tool.

A new person who logs into FolkMind can start with the most popular folksonomy terms and from that point browse related concepts or he can start by searching for a particular concept. Each node in the mind map can be tagged with additional information like a short description, its relevance (based on algorithms similar in principle to the one used by Google page rank), and additional information. This is the highest level of abstraction and at this level the user is more concerned with the connections between concepts than the individual documents that contributed to that concept. With usage the folksonomy gets richer and more concepts and connections between concepts emerge. Once the relevant concepts are identified, the next step is to drill down to the individual documents that pertain to that subject.

If the Internet can be considered as the virtual brain that represents the collective intelligence of humanity, then FolkMind is the pictorial representation on the same expressed as a mind map – hence the name.

How does FolkMind fit in as a Web 2.0 application

Let me explain how FolkMind application demonstrates the traits that are commonly observed in Web 2.0 applications. FolkMind can be a thick client that connects to the FolkMind server to retrieve its content but it uses the client side processing power for rich interactive UI and for local cache. As the user interacts with the UI, any change is reflected back to the server (this is similar to how Google Earth works). Though a user is given the option of marking certain connections as private, all other new connections and nodes that are created by the user will be stored on the server and will be visible to other users. This action is similar to a person creating a bookmark and tagging it using del.icio.us. Thus a user pursuing selfish interests (the motive behind creating a new connection or node is for his own benefit) build collective value for the rest of the users as a side effect. This phenomenon (also referred to, as the network effect) is critical to the success of a Web 2.0 application. As more content gets added to the system and as more users join in, the value of existing users will grow.

Conclusion

As we have seen in this article, the vision behind FolkMind is to be a powerful application with an intuitive, interactive UI that can harness the power of Internet by being capable of handing huge volume of data. Eventually, this will become the virtual brain of humanity!

References

What Is Web 2.0 by Tim O’Reilly
Personal Brain
Using Wikipedia and the Yahoo API to give structure to flat lists

Tags: Web 2.0  FolkMind

Updated: Fixed some typos and added a few tags

Community Server :: Forums (A .NET Open source discussion engine.), Community Server :: Blogs (A .NET Open source blogging engine) and Community Server :: Gallery (A .NET Open source photo gallery) all rolled up into one package called Community Server 1.0.
According to Rob Howard, they have started work on version 1.1 with the following theme.

Performance - we intend to run both www.asp.net/forums and blogs.msdn.com on Community Server. Today these sites serve millions of users on the same 2 web servers and single database server. We've introduced new architecture into Community Server and we expect performance and scale to be better overall, but we know we'll identify some new issues with these high traffic sites.
Simplification - We want to focus on simplifying the platform. We are going to introduce more wizard as well making the administration UI even more user friendly.
Quality - A big goal for Community Server was to release a product that was of very, very high quality. We are shipping with defects - actually about 50 that we know about - but most are minor and have acceptable workarounds. We fixed over 900 bugs (that we tracked). Our goal for 1.1 is to make the platform even better than 1.1

You can get it here.
Go Ahead and have fun!

CSS Zen Garden: A very interesting site highlighting the power of CSS. To quote..

There is clearly a need for CSS to be taken seriously by graphic artists. The Zen Garden aims to excite, inspire, and encourage participation. To begin, view some of the existing designs in the list. Clicking on any one will load the style sheet into this very page. The code remains the same, the only thing that has changed is the external .css file. Yes, really.

I have been using FireFox for quite some time and and am very happy with it.
It has got some really nice features for developers like the Dom Inspector that helps you to visualize the HTML DOM.
It also comes with a lot of extensions.

If you like this browser, please help spread the word..

Chris Pederick's web developer extension is a really nice tool. Some of the features are:

- Make live edits to the CSS of a web page..
- Validate the CSS, HTML and accessibility of a web page...
- Add a style sheet from your machine to a web page, etc..

Take a look at some screenshots.

I have finished writing a very simple demo application for my MSIL Parser porject. This demo takes an IL file as an argument and outputs the parsed data into the console.

I am using a compiled grammar file made by Goldparser. Currently there is a bug in the parser module of this application. When really big IL files are given for parsing, this module throws an
IndexOutOfRangeException exception.

Wile developing MFC based applications, I used a lot of ASSERT statements. But when developing C# applications, I see that very few people are using this.

For me using ASSERT macros was a way of enforcing 'Design by Contract'. May be we should have a better mechanism in C#.

I have a few ideas on how to do this with .Net attributes. I will write bout this shortly.

I had started a project called CIL-Parser on Source Forge. There was very little activity going on in this area for quite some time. Now I am trying to revive it.

The good news is that I finished the difficult phase of it - creating the IL grammar and testing it. I started off by looking at MS SSCLI implementation of the IL grammar. Fortunately the current version of dotNet framework SDK ships with the IL grammar (in the tools developers section).

I will be writing more on this in a few days...

As mentioned in my previous blog I was working on a Lotus Notes plugin for the Timex PIM software. I am happy to say that the basic code is now functional and I will be able to upload a version very soon. Pending Issues: Memory Leak - The PIM software expects the plugin writes to allocate memory using the operator 'new'. According to documentation, the software will internally free this memory. I think this is causing a strange bug (I am not sure about this). The Timex PIM software is compiled using and older version of MFC than what I use (for creating the plugin). So when the PIM software tries to delete memory that I allocate, it throws an exception and will result in a leak. I guess they should have used a library independent way of memory allocation like the CoTaskMemAlloc().   DSN configuration - The code that I wrote uses the ODBC driver for Lotus Notes to get the data. If I have to distribute the plugin, I will have to provide some utility to set up the DSN.

I have written an article on Codeproject that talks about a system-tray based clipboard utility called SmartClip. I have been using this tool for quite some time and I have found it to be very useful.
You can download it from here.

WARNING!!! This is an addictive tool :-)