Wade Arnold

OK, that was a bit of link bating. However for the types of problems that I am currently working on it really is!  I have been asked a couple hundred times in the last 8 months why I tweet about Scala and Hbase all the time. The larger post is coming but I figured I could answer the technical scala question first. In order to stop the comment flames let’s start by saying that I know PHP better than most human beings. It is something that I have been coding in since PHP 3 and have really enjoyed the language the entire time. I still believe that PHP is the best language for the web for the majority of programmers; it quite possibly has the best documentation and examples of any language like a DSL for the web! I am still slinging PHP and so is my company. I have been a speaker at lots of php conferences, aided in php’s documentation, enjoyed the community, and for the past three years have been Zend Framework committer. That’s all to say that there are lots of other PHP developers that are smarter than me; but I probably have met them.

So why is Scala easier? First it’s easier when you have more requests than you know what to do with. Fundamentally it’s easier because it does so much under the hood for solving concurrency issues in a syntactically concise manor. I have found that as I have moved from writing code that is designed to persist data to a system of service that consist of multiple UI and data service inputs and outputs with different hotspots and unique data mutations requires a different toolset. Partial functions, mutable and immutable types playing together, and scala’s collection library just make this simple. If the system grows just snap on a trait to add what you forgot.  Let’s take Twitter for example. Robey  Pointer wrote Kestrel, and probably others that I don’t know, that enables a lot of real-time scale for them.  Kestrel is a loosely ordered message queue that eloquently wraps scala’s mutable queue hashmap with a swarm of actors to attack the queue. Not to short change Robey as his guthub account is one of the best in the community! But the scaffolding of scala really makes this solution possible. The syntax of scala and ease of deployment also probably made it a reasonable goal to attack while #failwail’s were abound!

Obviously many people have dealt with this scale  problem in PHP successfully. I however find it very difficult to mentor people through the depths of hell that I have been through. When it comes to scalability in PHP this is what your resume looks like.

• Fantastic at PHP internals
• Amazing at Apache HTTPD and compiling appropriate PHP extensions.
• Nginx
• BigIP – More than round robin load balancing
• Intimately know how sessions work and probably write your own handlers
• Memcached
• APC
• AMQP
• BeanStalkd
• Code based sharding or at least master/slave logic
• C/C++
• Lots of security! It’s a problem with all dynamic languages.
• Zend Framework.

FYI the latest Zend Cluster Server really makes a TON of this easier! Bravo!

This is a really hard resume to replicate. I know; I have tried! More importantly it is very hard to debug. As we continue to add more screens, more protocols, and more data aggregation services, and frankly more data we need to be able use all cores in the server and the cluster. As Guy Steele implied at Strange Loop. The single processor programming model is going the way of the punch card.

What about bringing a scala programmer on board?

This is a scala resume.

• 6 months of scala
• Functional programming
• Akka Framework
• Lift Framework
• Nginx / Jetty

It really is a grind to overcome that first 4-6 months but if you need scale I would recommend taking the haul. It will change how you think about problems forever. There is nothing like having your entire system in source code that can be type checked in your IDE as you write. No need for amqp with actors, no beanstalkd with mutable queues, and it’s fast as hell!

If joomla and wordpress do everything you need, then your cool! Just like my lame blog! In the same vain I believe that PostgreSQL is the best database for the majority of data persistence scenarios. Hopefully my next post will be why HBase is the bomb!

Special thanks to the following scala studs for getting me past the grind. I really appreaciate the community support so far! Follow these guys on twitter and read their blogs. They are leaders in the scala community but have still taken the time to answer my neive questions! Almost everything I know has come from their books or blog posts.

• Daniel Spiewak
• Jonas Boner
• Debasish “The almighty” Ghosh < Debasish “The Holy” Ghosh
• Timothy Perrett
• Victor Klang
• Alex Payne
• Dean Wampler
• David Pollak

So is scala hard? Sure, but I personally recommend never bringing a knife to a gun fight. Especially when the competition has Uzi’s!

Wade

scala