In the old days, when programming in .NET you were signing yourself up to a lifetime of windows server, however things have changed.

Setting up your robots.txt file for your blog is easy, by adding a file called robots.txt in the root of your current themes directory.