Posted by linusl
Tue, 10 Jun 2008 17:32:00 GMT
Like most of the tech world yesterday, I was glued to the various live bloggers covering the Steve Jobs keynote. It was pretty boring with the hour-long fest of fruitless demos but in the end we finally got what we all were waiting for: the specs for the new iPhone.
Unfortunately though, I was extremely disappointed from the company that has blown my expectations year after year. Here are five reason why the new iPhone is NOT what it was hyped-up to be.
1. It’s not smaller
Though it’s not confirmed, it’s obviously not smaller than the first iPhone. If it were, don’t you think Steve would have mentioned this 50 times in his keynote? Yes, I understand he needed to make the phone last longer and put in a 3G antenna, but Nokia has a phone out right now with the same capabilities and is half the size.
2. It’s delayed
Why do I have to wait until July 11th? The phones are already made and sitting in warehouses in the US. I suspect it’s AT&T who is frantically trying to ramp up their systems. This is probably pissing off Steve a great deal but they could have done a better job on communication and planning.
3. Price and data plan
Yah sure the price is now an affordable $199 but the data plan is going to cost more. In the long run I am going to end up paying more and this was not touched upon in the keynote.
4. Camera is the same
So if the iPhone is the same size, the camera ought to be a little better. 2 megapixels is not going to cut it these days when the standard on most phones is 3-5.
5. More difficult to unlock
Besides the fact that it is running the new 2.0 iPhone version, AT&T is reportedly going to force activation at their stores. This clearly shows they are tired of people who are unlocking the phones and I am sure Apple has taken steps against this as well. Part of the fun of having an iPhone was the ability to add custom software and applications that were tailored to our needs.
Posted in General, Technology | no comments
Posted by linusl
Mon, 26 May 2008 20:06:00 GMT
This week I discovered a great (though very poorly named) program called God. It is a ruby monitoring gem that, as the author describes, “is like monit, only awesome”. It actually is pretty awesome because it makes monitoring scripts SO much easier to write versus monit. Plus it’s all in ruby so we can use all the regular ruby shortcuts without learning how to do them in monit.
The only drawback however is that God doesn’t not support monitoring. If you read the message boards and the blogs out there you can see that this is a feature that many have requested. However since the purpose of God is to simply start/stop/restart processes, the author of the gem has neglated to build in this feature.
I however have a work around to this problem. What you do is make a transition condition that monitors when the process is up and transitions it to a restart. The KEY here is to make sure that your restart command does nothing. For me I just have it sleep 0 which is close to a noop. See the code snippet below:
God.watch do |w|
w.name = “FOO”
w.interval = 5.minutes
w.start = “sudo /usr/bin/ruby FOO”
w.stop = “sleep 0” # stop command - HACK
w.restart = “sleep 0” # restart command - HACK
w.pid_file = “/tmp/FOO.pid”
w.start_grace = 5.minutes
w.restart_grace = 5.minutes
w.start_if do |start|
start.condition(:process_running) do |c|
c.running = false
end
end
# Hack: GOD doesn’t currently do monitoring. Therefore we need to run a
# restart command that doesn’t do anything. The w.restart_grace variable
# is the time in between checks, since after it does this transition, it
# will requery the state of the rails app
#
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 1.megabytes
end
end
Posted in Technology | 1 comment
Posted by James
Tue, 13 May 2008 00:44:00 GMT
A few days ago I wrote my first MySQL stored procedure.
For one of our applications, we had to make a big database schema change. As a part of this process, we needed to populate a new table with some existing data. However, there needs to be a bit of custom logic because the underlying data representation changed.
The algorithm for this operation is O(kn), where n is the number of rows in the original table and k is a small constant due to the way the data is stored in the original table. The unfortunate fact is that the original table has 3.5 million rows. So, I chose to do everything in a stored procedure.
Writing a stored procedure in MySQL is easy enough. However, the available tools suck. It took me forever to figure out that I have to change the delimiter from ‘;’ to something like ‘$$’ for everything to work. There is no debugger and no print statements. If you write something really complicated, it’s almost impossible to debug.
But the performance of stored procedures is amazing! I calculated that if I didn’t use a stored procedure it would take about 8 hours. When we ran the stored procedure, the whole thing finished in 15 minutes. It peaked at more than 13,000 queries per second! What’s more amazing, was that the application was still live and serving requests normally during the data migration.
Well, needless to say, I’ve become a fan of stored procedures, even though it was a pain to edit and debug.
Posted in Technology | Tags u | 1 comment
Posted by linusl
Mon, 12 May 2008 21:41:00 GMT
I am going to make another technical post this week. We are using Trac as a way track bugs in our applications. It is extremely popular which as a result makes it a pain to find information on Google (trust me, try searching and you will get hundreds of irrelevant results).
The other day I spent a considerable amount of time trying to customize the reports to the way I want them. One problem was how to create a strikethrough for bugs that were closed. Several people have asked for this on various forums but no one had a solution. I decided to write my own.
My solution was to create a new style in the stylesheet and its corresponding tag. Though I could have overridden the existing styles but I like to keep this as OEM as possible.
In /htdocs/trac.css Add the lines:
table.listing tbody tr.even_strike td {
background-color: #fcfcfc;
text-decoration: line-through;
}
table.listing tbody tr.odd_strike td {
background-color: #f7f7f7;
text-decoration: line-through;
}
In /template/report.cs add the lines:
<?cs if row.__strikethrough__ ?>
<?cs if idx % #2 ?>
<?cs set row_class=$rstem+'even_strike' ?>
<?cs else ?>
<?cs set row_class=$rstem+'odd_strike' ?>
<?cs /if ?>
<?cs /if ?>
Finally in our customer report:
SELECT
id AS ticket,
summary,
priority,
(CASE status WHEN 'closed' THEN 'true' ELSE '' END) AS __strikethrough__,
(CASE status WHEN 'assigned' THEN owner||' *' ELSE owner END) AS owner,
datetime(changetime, 'unixepoch') AS Closed, description AS _description,
reporter AS _reporter
FROM
ticket t
LEFT JOIN
enum p ON p.name = t.priority AND p.type = 'priority'
ORDER BY owner, priority, status
If you do this right, you should be able to see something like below:
Posted in Technology | Tags line, linethrough, report, strick, strikethrough, through, trac | no comments
Posted by linusl
Thu, 08 May 2008 18:35:00 GMT
This week, I’ll change it up and make a technical post. We ran into this problem when we tried to delete 20 million rows without bringing down our live
site. A simple ‘delete from FOO where id < 20000000’ would have taken too long because the database would have to go through those rows, recreate
the indices, and do all the internal SQL cleanup. Given these problems we had to do it a better way. The general idea is to:
- create a temp table
- insert the rows we want into the temp table
- recreate the indices
- switch over the tables.
The actual SQL statements is:
CREATE TABLE foo_temp (
`id` int(11) NOT NULL auto_increment,
`my_data` text NOT NULL,
);
INSERT INTO foo_temp (SELECT * FROM foo WHERE id > 20000000);
CREATE INDEX `foo_index` ON foo_temp(`my_data`);
DROP TABLE foo;
RENAME TABLE foo_temp TO foo;
Posted in Technology | no comments
Posted by Jeff
Tue, 22 Apr 2008 17:18:00 GMT
So what the heck is web 2.1??? First, the meaning of web 2.0 isn’t even clear, so how can you increment versioning on something that no one really knows the meaning of?
Well - here is a little history of the web that should qualitatively give you an understanding of what I see Web 2.0 is and what is now Web 2.1, which I will call The Viral Web.
The web as we know it today came a long way. First, you had Yahoo listing pages on one website like the yellowpages, then more data grew and Google started to make searching easier and more accurate. As for the content of webpages up to around 2000 were generally made by one person and then consumed by the masses. Examples of this are like personal home pages, online newspapers, and corporate websites.
Then the web started to evolve, things like Wikis (collaboration of many people for consumption by many people), YouTube (many people uploading videos to be viewed by even more people), and Facebook (many people customizing profile pages and uploading pictures to be consumed by even more people). In one sense, Web 2.0 involves giving users power to create content and using people’s social network to distribute information.
For a majority of websites, the discrimination is less clear - what do we make of websites such as eBay, where users put up items to be sold to other people. Even though eBay has been around for a while, it shares the characteristic of empowering users. There are many more examples in this area, and you can feel free to dub them whatever you want.
Recently, ‘Viral Marketing’ and ‘Viral’ growth have become the Fad of the Valley. Everyone wants their application, website, or plain old company to ‘Go Viral’. What is this really about?
Well this is really about harnessing the users ability to create content (such as in Web 2.0) in a different manner. In fact, the content that users create are actually messages sent to other users. For example, within the Facebook platform, we see this happening on many applications that attain millions of installations after just a few weeks! This is already happening outside of social network environments Ning has accomplished this by letting users create their own social networks.
The Viral Web involves startups taking the learnings from Social Networks and companies like Ning and applying the concepts broadly across other traditional internet companies. Having users participate in Viral Loops that the website uses is going to give the traditional internet companies a run for their money.
1) Essentially Nil marketing costs
2) Brand Names develop quickly
3) Further leveling the playing field between large and small companies (small companies can get big very quickly)
4) Lower overall cost structure of startups will make things cheaper in general for consumers (yay for us!)
The next few years are going to be quite exciting - get ready for the ride!
Posted in Technology, Business | 1 comment
Posted by James
Mon, 14 Apr 2008 01:40:00 GMT
Recently I’ve been doing work on scaling some of our medium sized web applications. In our case, the bottleneck is in the data layer, which is often the case for most web applications. And of course, scaling the data layer is always the most challenging.
Most people, instinctively think memcached when you tell them to scale the data layer. While memcache is great, it’s not the big hammer that everyone think it is. In most cases, memcache can speedup reads significantly, especially for data that is accessed repeatedly, however, it does very little to speed up writes, with few exceptions (as I will illustrate shortly).
The MySQL database in question, handles approximately 97% writes and only 3% reads. There are a few commonly known techniques for scaling this type of database.
- Group the writes in batches, thus doing fewer writes.
- Use insert delayed or update low_priority
- Insert into memory table and later batch insert it into the persistent table.
- Use master-slave setup and write to master and read from slave.
- Write a customized data layer to handle the writes asynchronously.
Obviously, these increases in difficulty as you go down the list. For our application, most of the writes can’t be batched. So we had to start with technique #2.
We really don’t worry too much about data consistency and transactions, because of the nature of the application, so insert delayed works just fine. The problem is that insert delayed and update low_priority queries only work with myisam and memory tables in MySQL. Rails by default uses innodb tables, so the tables in question must be converted. Depending on the size of the database, this could mean some downtime for the app. Additionally, myisam uses table-level locking, so we have to be really careful in how we use the database. This is where memcache comes in handly. I was able to cache the results of some commonly executed queries so they no longer hit the database, and the performance improved significantly. Using technique #2 along with memcache sped up our application 2 to 3 times.
The actual implementation was pretty simple. I wrote a Rails plugin to add a method save_delayed to ActiveRecord::Base. The are two problems with this. First the timestamp fields (created_at and updated_at) no longer auto populate. So, I added methods in the plugin to fill them out if they exist. The second problem is more serious. The records no longer automatically get assigned a primary key id when they’re saved. Because the save happens asynchronously, the id will always be 0. There’s no way around this except to not use the record after it’s saved. And if you need to access it in anyway, you should do a find and construct a new record object instead. This is okay for us, because we do a fire-and-forget save operation anyways.
The drawback to using myisam is that at some point, the table locking just won’t scale anymore. Innodb is a much better storage engine for larger datasets and we’ll have to move to technique #3. We can use insert delayed into a memory table, then have a process that moves data from the memory table to a persistent innodb table. This reinsertion process will have to be very well implemented because you don’t want to do deletes and inserts and start inserting multiple times or losing rows. I haven’t tried this, but will soon.
Scaling data layer is one of the most challenging part of my job. I love to see the application fly after making a few changes. This is partly why my job rocks!
Posted in Technology | 1 comment