All posts by wihl

Big Data

It’s Hadoop’s world. We just live in it

It’s Hadoop’s world. We just live in it. Welcome to #hw2011!

That was the starting battle-cry from Mike Olsen, CEO of Cloudera as he kicked off the third Hadoop World Conference. Indeed, after drinking the kool-aid for two days, I’ve been almost fully ingested, stored and transformed even if I have yet to be accessed, let alone managed.

For it seems that within a few years, all digital information, including my electronic Freudian id and perhaps my ego as well, will be deposited into Hadoop, forever ready to be accessed via any number of different social and structured graphs.

Hadoop background

Some have speculated that within five years, Hadoop will hold 50% of the world’s information. I now believe that to be true, albeit potentially a copy of the other 50%, if not uniquely in Hadoop.

Hadoop and its Google ancestors enable storage on a scale and scope never previously possible with baked-in redundancy and resiliency at lower operational cost than big iron solutions. And the software is free, needing nothing more common commodity hardware and a Java host.

Google created and shared the concept. Doug Cutting and a colleague started an independent Apache licensed implementation five years ago. Since then, it has been adopted by the largest web properties: Facebook, Twitter, eBay among others. Even major enterprises like JP Morgan and Disney have been using it in production for at least two years.

Commercially supported releases are available from Cloudera and Hortonworks.

The Conference

Hadoop is still in the early adopter stage and has not yet crossed Geoff Moore’s Chasm. This is most reminiscent of the state of the web circa 1994. Forward looking companies are making incredible strides in competitive advantage using primitive tools and smart developers.

Cloudera is doing great job in championing the ecosystem. They recognize that growing the overall market and adoption is the correct long term path to riches. I look forward to #hw2012.

Windows Server

Unshackled Microsoft Declares War

[Image via Business Insider]

Microsoft Build was touted as the most important Microsoft developer conference since 1995. Believe that hype – Microsoft is alive again, unshackled by the DOJ Consent Decree that expired in May 2011. Windows 8 is only the beginning of a multi-front war Microsoft will engage in across the entire software industry.

A smiling, upbeat Steve Sinofsky portrayed a positive message throughout the overly long keynote, but the undertone was more Churchillian:

“We shall not flag or fail. We shall go on to the end. We shall fight in tablets, we shall fight on phones and servers, we shall fight with growing confidence and growing strength in the cloud, we shall defend our island, whatever the cost may be. We shall fight on the social networks, we shall fight on the desktops, we shall fight in the browsers and in the developer communities, we shall fight with interns; we shall never surrender.”

The breadth of Microsoft’s ambition is refreshing and revitalizing. The scale of Windows 8– from phones to enormous servers–is just the beginning. Microsoft’s subtle agenda is a seamless integration with the cloud. This leverages Microsoft’s strengths of being one of the very rare companies capable of making an end-to-end experience from client to corporate server to cloud under a single set of APIs, management tools and security policies. Microsoft knows and understands that integration has always been and will continue to be the key to leveraging their platform. Corporate customers have the money and desire for this integration, rather than a hodgepodge of iPads, third party security apps and inconsistent tools.

Chris Jones presented the LiveID integration. I met Chris back in 1997 at the private developer launch of IE3. IE3 was the first version of IE that was finally superior to Netscape Navigator, both in execution and extensibility. It’s exciting to see a veteran come back to attack another Silicon Valley competitor. Competition is good for the industry and the economy in general.

Let the war begin!

Programming

Robert Morris, Encryption and the Pony Express

[Image via NexGadet.com]

Robert Morris Sr. died June 26, 2011. I had the honor of meeting Mr. Morris in 1996 at a Dartmouth Workshop, where he gave an interesting and dynamic talk on transportable agents. He then took questions from the attendees. In my ignorance, I asked him:

“Aren’t you worried that the vast majority of common encryption schemes (SSL, RSA, etc) are based on prime factors of large numbers? If someone found an algorithm for this wouldn’t all common encryption be useless?”

Little did I realize that speculation has been rife for a long time that the NSA had in fact solved this long ago, either by math or vast amounts of hardware or a combination of both. As the former Chief Scientist of the NSA, Morris had obviously been asked the question before by people much smarter and with much more at stake than me. He could have easily brushed it off. Instead, with the infinite patience of parent, he carefully chose his answer. Continue reading Robert Morris, Encryption and the Pony Express →

Events, Social Media

Are Tweets Another Way of Saying I Love You at 300 baud

Yesterday I attended the very useful MIT CIO Conference. For the first time at a conference, I was one of those a live tweeters, using HootSuite on my iPad. Unlike that curmudgeon Bill Keller,  I did not experience anything akin to crystal meth. Tweeting along the way enhanced the conference experience significantly.

The most enjoyable part was sharing thoughts with others in the room in real time. I was able to see the emphasis on parts of the dialogue that I would have otherwise missed. Based on retweets of my own messages, others likewise found value. Being constantly tuned in helped me keep my focus even after hours in a darkened theatre.

The immediate feedback from presenters was a nice surprise. To comment about a presenter and then get an individual response three minutes after the session is new and exciting. Continue reading Are Tweets Another Way of Saying I Love You at 300 baud →

Mobile, SharePoint

Suspending Development of our BlackBerry client software for Microsoft SharePoint

On May 6, 2011, we suspended development of Pylon, a BlackBerry client for Microsoft SharePoint. This was a very difficult decision, as we started working on this product over three years ago and invested significant R&D, sales and marketing effort into it. Three issues factored into the decision:

  1. Breadth of Features: Microsoft SharePoint is a large and multi-dimensional product. As we narrowed the feature set to appeal to a few customers, we grappled with how to scale to a broader base. Enterprise SharePoint customers frequently build solutions assuming that Internet Explorer will be the user interface. Bringing these custom solutions to multiple mobile environments would require significant engineering work from both SoftArtisans and the customer.
  2. Microsoft’s SharePoint Protocol Patent Licensing, which imposes a significant royalty on a product like Pylon: By our reading, a client-only, software-only product like Pylon would require a minimum royalty payment of $5.80 per license, which is unfeasible in a typical app marketplace where products are either free or typically less than $20, not counting an app store’s 30% fees. Continue reading Suspending Development of our BlackBerry client software for Microsoft SharePoint →