Archive: IZEARanks
Last week we move the IzeaRanks calculator script onto a new server. We also deployed some changes to the system to better synchronize data between the IzeaRanks database and PayPerPost.
This morning I noticed that the script didn't run over the weekend. I still have yet to find out just why that happened, but I am manually running the script for the last few days to update everybody's ranks. I expect to be done by the end of the afternoon. Until then though you may find your rank in PayPerPost is not current.
Sorry for the inconvenience
Pete
As I pointed out in the earlier post, we were having problems with IE which resulted in IE not sending us the data on Windows based visitors (hence low numbers). In fixing that though we seem to have exposed a really nasty bug in Internet Explorer itself.
Basically the ITK code needs to be placed in the body of the web page on your site for the bug to not rear it's head. We think that's unacceptible and are working out our own solution to Microsoft's problems. In the meantime we're rolling back the ITK code to the earlier version so that the crashes IE users are experiencing go away.
The Alpha Test of IzeaRanks, and tracker engine that feeds it, has gone really well. We’ve had a very high uptake within izearanks.com, and most importantly of all we’ve had some great feedback.
By far the most pressing issue has been with the accuracy of the data that we track and report. Lots of people have said that our numbers are significantly lower than those reported with other analytics tools like Google Analytics and SiteMeter. We’ve worked very hard to identify why this would be the case and come up with solutions. I’m happy to report that the solutions are now live, and you don’t have to do anything at all to update your blog to receive the new script.
First, the previous version of the tools waited for the entire page to load before it started to send us data. However, some of you have blogs that take a while to load. You might have a lot of content for example, or perhaps you have a lot of third party plug ins on your blog, all of which add to the load time. The end result is that visitors to your blog may move on to the next page before the page has finished loading. As a result, the ITK never fires any data to us.
We’ve changed all that. First, we reduced the size of the ITK by 25%. Later today we are also going to compress it, resulting in a reduction of 85% in total from the old version. Second, we changed the code. The new code now fires data in two stages. As soon as the script is loaded, data gets fired at us to record the site visit and page view. Then, when the rest of the page finishes loading the second part fires letting us track activity on your in and outbound links etc. The end result is, of course, more accurate visit recording.
We’ve also got some plans in place to speed up the physical delivery of the Javascript to your readers, but I’ll talk about those in a later blog post.
We also noticed some serious problems with the way Microsoft’s Internet Explorer works with our script. First, Internet Explorer would not recognize the name we gave to the embedded form in your blog. The Javascript creates an embedded form which it populates with the data to fire at the tracker server. Since IE would basically ignore the name we gave the form, the ITK would fail to fire any meaningful data at the server. Obviously this caused the bulk of the low traffic complaints we got, and we’ve fixed how we work with forms in IE now. In addition, we’ve changed the way we batch up the information to send to the server to make it more compatible with blogs that use XHTML instead of plain old HTML. The old way didn’t work too well with XHTML blogs and so, once again, we got lower numbers than we should have.
Finally, on the server we had issues where people with ultra high security settings in their browser caused the ITK to fire blank data at the server. The server, stupidly, didn’t know what to do with the blanks and ignored the entire record. Once again, that meant more low numbers, and it’s been fixed.
I’m still looking at an issue relating to synchronizing blog records between IzeaRanks.com and PayPerPost.com and I’ll be blogging about the work on that when it’s ready to go. For now, I hope you enjoy the changes we’ve made to further improve the ITK and move us a lot closer to getting this thing out of Alpha Test.
I've been a little quiet lately as Jamie, Daniel and I have been beavering away on the remaining issues. Those are that some sites still have no rank when they should, that the numbers are low across the board, and that the ITK needs to fire and process a lot quicker for those of you with slow loading pages.
We're almost there and have identified a number of fixes that need to be deployed. There's still some code to do but we're close to making a new release.
On the low numbers front, the problem is more widespread than I originally thought, but it's impact is a lot less than originally perceived. Since the numbers are low across the board, everyones relative rank (i.e your position relative to everyone else) is actually very accurate. When we roll out the low numbers fix I don't expect to see a lot of jumps in rank, since the whole network will get more data.
More news as I have it, but rest assured we're not sitting idle on these things.
Thanks for your patience.
I deployed the latest fix to the ranking engine last night and at last it's fixed a number of outstanding bugs with low numbers and zero ranks. It also addressed an issue with uniques that we weren't aware of before yesterday.
Basically a database change a few days ago meant that uniques were not getting recorded. THis is why, for most blogs in the system, you'll see a dashed line on the uniques graphs. Dashed lines show up when there's not hard data for a variable and so the system infers what it thinks the numbers should be based on trends. So, the upshot of that data loss from a ranking perspective is actually minimal. It's great to see those numbers coming back in.
The low numbers that were getting reported (people stating that the numbers we report are lower than other analytics systems) was also related to the latest outstanding issue with zero ranks. The ranking engine expects URLS to be passed to it in a certain format. However, we found a number of browsers/blogs/blogging engines (we're not sure which it is) passing in fully escaped URLs. The ranking engine failed to identify these records as a result. This has now been fixed and I notice this morning that the vast majority of sites that had zero ranks now have a rank, and the numbers for everyone else have gone up accordingly.
There will still be some differences though between what we report and what the other analytics engines report. This could be due to a number of reasons (differences in loading time between the various engines javascripts is one reason), but a prime candidate here is date and time. If you visit Google Analytics and look at yesterday's data for a site, what actually is yesterday? Is it yesterday your local time? Probably not since this could easily mean that you could hop on a plane, fly to a different time zone and see different numbers. Is it yesterday on the visitor's computer? Probably not again, since yesterday's figures could change if you're viewing in a time zone way ahead of everyone else.
Most often the analytics engines choose either local time on their servers, or UTC. As with the other Izea properties that have a time element to them (PayPerPost.com for example) we use Eastern time (Orlando time). So, there's a strong chance that we'll report yesterday as 3 hours less than a system in California, so we'll have lower numbers for yesterday, but include some data in today's figures which the other engines would consider as yesterday.
There are still likely to be some differences, and given the Alpha nature of izearanks their may also be some bugs in the ranking engine. I'll continue to address them as I find them. But today the system is a lot better and lot more accurate than it was yesterday (our time ;) )
We've been feverishly beavering away fixing issues with izearanks here in the office. Although we ran testing on the system for over a month,and included some of the user community, before we went live, it's perhaps almost inevitable that some bijou bugettes slipped through.
I deployed an update last night that fixes many of the problems PayPerPost users were having with a 0 rank appearing on sites that were actually getting traffic. To cut a long story short, if a user changed their blog url in PayPerPost, the ranking engine failed to update with the new url. It then tried to find tracking data for the old url and, finding nothing, assigned a zero rank.
I'm currently running a database migration for the next issue. Some people have raised a concern that they have a zero rank after registering their blog in izearanks.com. The issue seems to be related to the way the Izea Toolkit is passing encrypted ID's back to the server. There's a data type mismatch on the database field where the data is stored, and so updates are failing to happen. This is only for some sites, and even though only those registered actually through the interface at izearanks.com.
The database update is running now, the new server code has been deployed and as soon as the database update completes the server will fire up and start collecting the data it had been failing to. This means those newer izearanks.com sites with no data reported will start collecting data shortly and should be assigned a rank when the engine runs at midnight.
I am pleased to see that people are starting to use the IZEARanks API to develop their own tools. Dew Knight has deployed the first widget that I know of, a small badge that displays a blogs RealRank. I like this idea and can't wait to see what else you guys come up with. If you are using the API to develop something cool let us know!
I've been digging through the issues related to some of you having no ranks and I have identified a couple of reasons. The first is not really a bug, so I'll explain that one now.
We collect data on your blogs 24/7, but the real rank calculator script runs overnight, just after midnight. If you register your blog today, you are not going to see a rank for your blog until after midnight. That's also the point in time that data from the tracking system is moved into izearanks for graphing. So, add a blog today, and check back tomorrow to see what happened.
The second issue is a bug and I'm working on it now. Many of you reading this have PayPerPost blogs, and some of you recently asked to change your blog urls. The problem with this is that the rank calculator migrates blogs it hasn't seen before out of PayPerPost each night, and into the ranking database. So, your blog was migrated already, with it's old URL.
The Izea ToolKit though is now sending data to the tracker from your new url. PayPerPost is aware of your new url. IzeaRanks on the other hand doesn't pick up the name change to your blog and so when the ranking script runs it searches tracker data for your old URL. Obviously, it won't find any data relating to the old URL and so you don't get a rank. The problem is further exacerbated when someone changes their blog url in PayPerPost, then goes and adds the new url manually to IzeaRanks. Now we have two urls which are different, but effectively the same site in IzeaRanks. The new one, the manually added one, gets a rank the day after it's added but obviously that change won't propogate back into PayPerPost.
I'm working on automated tests for these things now, and solutions to them and hope to deploy them tomorrow during the day.
We released IzeaRanks.com yesterday and the uptake has been very strong. I hope you guys are enjoying our latest code-baby.
There is an issue that's causing some concern though, and that is that the numbers for visitors, page views and so on are lower than they should be. I'm taking a look into this issue myself later this afternoon to see what the problem is.
We have a team focussed right now on working on other issues with the site,but I wanted to bring special attention to the numbers thing because it's a potentially big deal.
I'll keep you all updated.
Pete
Friday we very quietly rolled out a change to the RealRank calculation engine that we've been toying with for some time.
Many of you noticed that your RealRanks fluctuate a great deal from day to day. That's because it's a 'real' rank. As more blogs get added into the calculation there is a strong chance you will change position the next day. Similarly, if a few blogs above you have a bad day you can easily leap a number of places on the rank.
While this is all great for accurately showing a daily rank, it's not so great for continuity, spotting trends within your blog and so on. It also unfairly punishes great traffic blogs that just have a bad day.
Sitting underneath the rank is a score. Each night we calculate this score for every blog we're tracking. THe resulting scores are then sorted to assign each of you a unique rank. There is only ever 1 number 1 for example. It's the daily fluctuation of this score that causes ranks to jump and dive day to day.
The change we made fixes this. Daily scores are still calculated, but to get the ranks we take the previous 7 days scores for the blog, average them and then rank the results from 1 to n.
From the sites I've looked at in some detail, this works really well. It smoothes out the rank over time, and it also prevents great blogs getting slammed on a single bad day. We're going to stand up the real ranks site towards the end of the week where you'll be able to view and work with graphs of your site's ranks over time. When we do this you'll see how the changes make spotting trends in your rank very simple.
