One of the many websites I have and one of the most popular according to Alexa (it’s currently the 6707th most popular website in the world?) is XML-Sitemaps.com. This website is slightly different for me too as it is the only website I have which is a joint venture with another person. I can’t really take any credit for this site as the brains behind it is a very talented programmer named Oleg Ignatiuk.

I first came across Oleg at Scriptlance.com which is a site that puts programmers and webmasters together. The idea being if you need some programming done on a website you setup an auction and programmers will bid on the project for how much they will charge for doing the works.

When I first heard about Google’s sitemap program, not being that technical I asked Oleg to take a look and see if I could use it for one of my websites. It turned out that I couldn’t as the site I had in mind was dynamic in that all the pages were written on the fly by the server so they didn’t really exist. The way Google’s sitemap generator worked is that it would scan your file system for files and add them to the sitemap. As these files didn’t exist I couldn’t do this. The other option would be to scan your log files which would work but only for pages that people had visited so potentially a large volume of pages would be missed.

So the next question was, can you write me something which would work. The result was a truly amazing script which would crawl my website picking up all the pages and adding it to a sitemap which I could submit to Google. Oleg went the extra mile and added in some other useful features for me like identifying problem areas of the site where there were broken links and reporting pages which these were on so I could easily fix them. He also made it possible to create multiple sitemaps automatically. Google only allows 50000 urls per site map. Because my site had around 300,000 pages it was necessary to split up the sitemaps and create a sitemap file for the remaining sitemaps. This was all done automatically and all I needed to do was press one button to start the crawl.

Now I had this amazing script which worked perfectly and I remember looking around the internet for something similar and there wasn’t anything that even came close to it. So I asked Oleg what he thought about selling it to the masses. There were so many webmasters in a similar position to me that I was sure there would be a good demand for a low cost script that enabled them to create Google Sitemaps for their own sites. By this time the script had cost a considerable amount of money to develop. The decision to sell it was two fold. 1. To recoup my development costs and 2. to reward Oleg for all his hard work. We agreed I would pay for development costs and hosting and promote the site and Oleg would handle all the technical support and we would split any revenue from the site.

We decided to create a free online sitemap generator that was limited to 500 pages so most webmasters would have an easy solution for creating sitemaps and hopefully would spread the word around about the site. It was launched on 25th June 2005 and was generating quite a lot of interest within days mainly thanks to Google who linked to the site from their Sitemaps 3rd party programs & websites page. Meanwhile Oleg worked hard on adapting the script to make it suitable for self installation and to protect the code. We launched the standalone php script on the 5th July 2005. The timing couldn’t have been any better.

I remember waking up on the 6th July 2005 and checking my emails and thinking “that’s a lot emails to come in overnight”. On opening my in box I was surprised to see order after order after order for our newly launched sitemap script. I was amazed and it took a while to realize where they were all coming from. It turned out that someone had written an article and mentioned our site in it. This was picked up by Sitepronews.com and sent out as an email newsletter to all its subscribers.

The site has gone from strength to strength since. In a little over a year it has created more than 190,000 free sitemaps crawling almost 29 million pages. It now creates more than 1000 free sitemaps a day and had to be moved to a more powerful dedicated server to cope with the demand. The script has evolved too with added features due to customer demands such as creating text sitemaps to submit to Yahoo! and html sitemaps for human visitors.

I am very pleased with the way this site has worked out and its success is predominantly due to Oleg’s dedication, skill and hard work. I am obviously biased about how good this site is but check out what other people are saying.

Posted in My Websites