When you reach the giga, peta, and exa orders of quantities, strange new powers emerge. You can do things at these scales that would have been impossible before…
Kevin Kelly has talked about the coming age of data, oodles of the stuff thanks to the Internet and what we’re doing with it. Here’s a nice video visualising how all this data and the devices connecting to it will define the future, albeit at the scale of trillions rather than zillions…
…and the makers of the video have more details on their web site – MAYA Design – including a research paper for download (PDF).
Hot on the heels of announcing the new PowerPivot for Excel at the SharePoint conference recently, Microsoft’s Live Labs have announced another new tool with a very similar name: Pivot. So it seems we have PowerPivot for Excel and Pivot for the web.
There’s no mention of whether or not the products are technically related or just share the same name. What they do have in common is the goal to easily visualise massive amounts of data.
Side note: I wanted to embed the video (you can see it on the Pivot web site – link below) but the embed code provided didn’t work. Wish Microsoft could use a simple standard for embedding like everyone else…
So much for publishing soundbites during the recent SharePoint 2009 Conference. What can I say, I got distracted 🙂 Here’s a delayed one.
Microsoft announced a new product at the conference that has been going under the codename Project Gemini – PowerPivot
PowerPivot is being released as an add-on component to Excel 2010 (licensing not available at time of writing). It’s purpose: analysing massive sets of data using familiar tools. It brings business intelligence (BI) into Excel. Historically, to do such large scale analysis has required specialist tools.
PowerPivot enables incredibly fast filtering and sorting of spreadsheet data extending to 100 million rows. That’s a pretty big dataset for Excel to handle. PowerPivot includes some nifty compression algorithms and the working data set is read only. There are features to enable you to edit related tables that feed into it. With SharePoint 2010 you will be able to display the content and analysis in web parts for browser-only scenarios. And whilst its title suggests it’s a giant PivotTable, PowerPivot is not your traditional Excel pivot table. You can have multiple slices based on related tables to cross-analyse the data. Here’s a couple of images taken from the conference:
In the image above there is the main pivot table (selected in blue) summarising and filtering total purchases by selected continents. To the left and top left of it you can see two slices that are being used to further filter the data by genre and rating
In the image above you can see one of the new functions that are included with PowerPivot, it is creating a sum by matching values in a related table. It’s hard to visualise how different this is to the traditional Excel pivot tables and formulas. One example given during the talk was that PowerPivot could enable historical comparison analysis such as comparing accounting information across financial years.
This is an interesting move for Microsoft as we enter an era where massive amounts of data are being created and shared across the Internet. Finding easy ways to visualise such quantities of data is a hot topic. Microsoft is not the ony one coming up with new tools…
Some reading on how massive amounts of data is challenging conventional wisdom:
- Recent Innovations in method (The Technium, December 2004)
- The Google Way of Science – replacing the hypothesis (The Technium, June 2008)
- The Future is Big Data in the Cloud (GigaOM, October 2009)
Side note: Whilst I can see where Microsoft got the name from, I can’t help but keep calling it PivotPoint instead. Blame SharePoint, PowerPoint and PerformancePoint for that 🙂
Last week I attended the annual Technology Management Symposium at Downing College, Cambridge University. One of the hot topics was energy efficiency and there was a lot of talk surrounding OLED displays.
One of the keynote speakers explained how OLED displays can be incredibly efficient because to display black pixels requires minimal power. However, those benefits are currently lost when using the displays to browse the Internet. Because the default colour scheme is black text on a white background. (This site being a case in point.)
Will we one day see promotions encouraging the use of black backgrounds for web sites? (I may start an experiment…) And will it also mean the return of the green text? People really will start claiming the arrival of cloud computing marked the return of the mainframe 🙂
- Institute for Manufacturing Annual Technology Management Symposium
- OLED on Wikipedia (OLED = Organic Light Emitting Diode)
Am in the process of sorting through and tidying up the library (which means broken links galore in old blog posts but will try and fix those). In the past, some items have gone straight into the library without a blog post. To fix, I’ll be posting those which are still relevant over the next month. Here’s the first…
The Internet Paradigm: Talk by Tim O’Reilly in July 2003, at Microsoft UK
Tim O’Reilly visited Microsoft UK in July 2003. Here’s a short summary of his talk. The talk was based on a theme that was to later become known as Web 2.0. Links to additional information included at the end:
Paradigm shift #1: Hardware
Prior to 1982, hardware ruled. Then IBM released specs for building PC computers. Didn’t seem that important to the industry at the time, they were ‘toys’. Took a decade to really take off, but along cam Dell, Compaq bought DEC…
Paradigm shift #2: Software
Software was now decoupled from hardware. Lock-in and competitive advantage moved to software. IBM had given away the future… to Microsoft. Hardware became a commodity.
Paradigm shift #3: The Internet
Applications and information decouple from both hardware and software. Lock-in and competitive advantage moves to the data and customer relationships. Software becomes the commodity,.. Think Amazon and eBay – the application will stop working without people. It’s the participation age.
Where is Linux really successful? Not as a traditional operating system. It’s Google, Amazon, Yahoo. (and you don’t get access to their source code, only the APIs, sound familiar?)
3 trends that matter:
1, Software as a commodity:
- Amazon switched from Unix to Intel to save costs (10x saving running Linux on Intel)
- Apache means web serving is not a revenue opportunity
- MySQL threatens to do the same to databases
- Software is built for use in delivering services
- Internet-era apps are updated daily, not yearly (e.g.
http://www.salesforce.com/ = 12 updates
between 1999 and 2003)
- ‘Info-ware’ – interfaces built with dynamic data, scripting rules
3. Network-enabled collaboration
- ‘Ad-hocracy’ – people just get together to fix things, distributed internationally (skills, costs, timezones), like-minded devs find each other
- Power shifts from companies to individuals
- Users help build the application
More people have contributed to Amazon than have contributed to Linux…
Small pieces loosely joined:
- An architecture of participation means that your users help extend your platform
- Interoperability means that one component or service can easily be
swapped if a better one comes along (e.g. Google data centre)
- Lock-in occurs because others depend on the benefits from your service, but you are not in control.
What does this mean for Microsoft?
Got to change at some point, not going to see the same margins as in the past (IBM had to get used to this one). Could MSN be a big part of the future? Currently focused on consumer, but what about business?
- Open Source Paradigm Shift (blog post by Tim O’Reilly in April 2004)
- MSN realigns with Windows Platform (MS press release, September 2005)
- What does Web 2.0 mean (Joining Dots blog post, October 2005)
Filed in the library under Talks
Stop the clocks, blogging has recommenced 🙂
Couple of cheat posts coming up, starting with this one, which are really reproductions of comments I’ve left on other posts but with added juice.
Henry Blodget posted the following article on Silicon Valley Insider: It’s time for Microsoft to face the reality about Search and the Internet
It’s a great article and worth a read. Here’s the comment I left there:
I think people make a good point in highlighting that competition in search is a good thing for us as consumers. Just not sure it’s a good thing for Microsoft
Comparing with the likes of SQL and Exchange is comparing apples with pairs*, I always tell people to never underestimate just how hard MS will work to develop the winning product. But past successes have always been about bringing in a lower cost product with good enough features to compete against an expensive market leader (the business intelligence and systems management markets being two of the latest focus areas gaining ground in the enterprise software market).
Competing against ‘free’, a product used by one audience and paid for by another, is a completely different challenge and one that MS has yet to succeed in. Time will tell. But I doubt it will come from competing like for like. Google didn’t knock Alta Vista off the top by copying their business model. To take over a market means to do something different that weakens the incumbent players. Adding to the challenge is that ‘free’ or ‘freemium’ models have yet themselves to stand the test of time. Somebody somewhere always has to pay, one way or another. Making money from sales of a product or service still have far more long term potential than making money from people paying for the attention you’ve managed to capture.
And that all said, I still wouldn’t underestimate MS, Google isn’t the only one who can create ‘waves’** under cover
I suspect the ol’ Google vs Microsoft debate will rumble on for a few years yet. Steve Ballmer and quite a few influentials within Microsoft would like a big slice of the advertising market that is a fair bit bigger than the software market. But I’m still not confident that’s the right goal.
The argument goes that people are becoming used to not paying for online services. Yet Flickr has done quite well getting people on to premium accounts. Virtual worlds and multi-player online games like World of Warcraft also seem to find plenty of paying customers (I’m one of them, and a girl too – take note, Xbox team. I’m in that market that Nintendo noticed whilst considered a has-been at no.3 in the console market a few years ago). I’m also in that apparent small minority who pays for their music online. Amazon has done quite well just selling stuff that you go looking for rather than have thrown at you in a glitzy banner, eBay isn’t doing so bad either. None of them dependent on advertising revenue. The last two examples have both made money from closing the distance from customer to seller without advertising interrupting the process.
And who wants to be an advertising company anyway? Google succeeded ‘cos they managed to make the ads as unobtrusive as possible and came up with a great revenue concept (auction the search words) to get companies competing for what little ad space there is. Most people seem to dislike ads unless it is for the exact company they are looking for (I do a search for Hilton Hotels, I want the damn official web site, not a million travel web sites) or something completely original (that ends up in the ‘top 100 ads’ TV chart). Even worse are the fake web sites that get into search results only to present you with a page of even more ads than the search engine dared to show you.
Advertising is not a loved market. Microsoft is not a loved company. I don’t know if that’s the synergy they see but it’s not a great start. Without doing the research, I’m guessing the margins in advertising are not as healthy as software. And Google may be raking the cash in from ads but is pouring a fair chunk back out again, not least running the hardware to support YouTube.
What is right is the desire to create a new market. Monopolies (natural or otherwise) and market domination rarely last for very long… unless funded by government but let’s not go there today. Microsoft needs to keep testing new waters and best do it whilst there’s oodles of cash in the bank than start when it’s running out. And call me biased because I used to work there, but I still hope all this Google chatter is smoke and mirrors whilst they work on something worth paying for.
* Oh dear, didn’t notice I’d used pair instead of pear when posting the original comment, and you can’t go back and edit them. Oops.
** Google cleverly making waves (Techcrunch)
You may or may not have noticed, Google has been rolling out some online services that challenge one of Microsoft’s key territories – Office. Google’s online version is Google Docs. You can use it to create documents, spreadsheets and presentations using nothing more than a web browser with an Internet connection.
Currently, Microsoft doesn’t appear to have an online version of Office available. If there is one, I haven’t found it. But searching for variations on ‘Office’ gets all sorts of junk. Tip for Microsoft. When it comes to having a version of Office available online, consider using the following URLs – http://www.microsoft.com/docs and http://docs.microsoft.com/ 😉
Here’s the interesting part. I’ve been playing with spreadsheets for a couple of ideas and one feature required me to delve into Google Docs for the first time. Whilst Excel 2007 was best for one solution, Google Docs was best for the other. That, I didn’t expect to happen.
Case 1: Excel 2007
The image below shows the Googleshare (What % of results containing one word also contain another) for various keyword combinations. The legend shows each primary word, the x-axis shows the secondary words, and the y-axis shows the percentage value for each combination:
Unsurprisingly, Apple gets a lot more love than anyone else. It gets more ‘bad’ too but that might be caused by references to the edible edition.
The data was created via a web query against Google. It’s actually a lot easier to create the data connections in Google Docs than in Excel, thanks to Google’s importXML function (it’s the reason I started trying out Google Docs), but Excel was better for playing around with the data once it was imported. The standard charts built in to Google Docs are limited in features and duller than dishwater. Net result for me – I’m now using Google Docs to test word combinations and the ones that generate interesting results are then reproduced in Excel to further analyse and visualise. (Side note: tried to do the same with Live search results but couldn’t find a way to extract the data)
Case 2: Google Docs
The image below shows what you earn in interest when investing £1,000 for 5 years in an ISA versus a normal savings account (ISAs have higher interest rates and are tax-free):
Except, it isn’t actually an image, it’s an interactive chart provided by Google Gadgets, that hook directly into Google Docs. And on top of that, I published the chart enabling it to be embedded here without anyone needing Google Docs to view it. Click Play to watch the visualisation. If the bubbles don’t change size, Click ‘Size’ on the right and select ‘Interest’. (Haven’t figured out a way to pre-configure the settings.)
You can’t publish elements from Excel to the web without using SharePoint as the web platform. And Excel doesn’t have currently have any funky interactive visualizations, like time lines and motion charts. In Google Docs, you simply select ‘Insert’ and choose ‘Gadget’ to access a range of interactive visualisations. And to publish them, click on Publish and you get a snippet of code that you can insert on any web page, no specialist platform required.
New ways of working with information
Whilst I’m not surprised that Microsoft is not keen to provide an online version of Office just yet – they have a revenue stream to protect whilst Google doesn’t – avoiding the inevitable seems to be a riskier strategy. The surprise from playing with Google Docs has been discovering features it has that the current version of Microsoft Office can’t do.
Google has its challenges too – like finding a revenue stream for Google Docs. Unless relying on advertising revenue generated from the search side of the business is considered enough… that thought should freak out Microsoft. What if Google Docs is more about ‘brand Google’ than revenue?
Online vs Offline
One of the big arguments against using browser-based services has been their lack of offline capabilities. You typically need to be online via an Internet connection to access content. This is beginning to change. Google has recently announced that you will be able to take Google Docs offline. And they are by no means the only service provider planning to go down this route.
But here’s the rub. I hate using the browser offline. I don’t know if it’s just me or if others feel the same. Working within a web browser is all about being connected to the Internet. I accept certain limitations in the application, such as: a fixed-size input area like the one in Blogger that I’m using to type this post (ditto for writing emails); clunky in-page menus with limited options; missing features. I don’t want to take those limitations with me offline. When I need to be offline or use advanced features, I want proper applications. Even for reading blogs. The one application that is never opened when I’m on a plane is the web browser.
What should Microsoft do?
Three things I would be bleating on about if I still worked there:
1. Provide common gateways between products and online services
Help is already integrated (you can download templates etc. from Office online). Now it needs to be at a natural extension for all features. ‘Insert Chart’ should include the option to go online and access new Chart types, as they become available. If Microsoft is serious about their ‘software plus services’ strategy, all products need the ability to interact online and integrate incremental enhancements. The motion chart used in this post comes from Google’s acquisition of Gapminder Trendanalyzer – see end of post for a much better example. Microsoft has demonstrated a similar technology at its annual TechFest (where MS Research shows off their projects) – DynaVis. Goodness only knows when we’ll get our hands on it. Holding features until the next product release is no longer a good strategy…
2. Make it easier to share work created in Office on the web
And that does not mean SharePoint. At the moment, to publish elements of Excel online requires SharePoint (and not just any old SharePoint, the most expensive version – Microsoft Office SharePoint Server 2007 Enterprise Edition). Not necessarily a problem for collaborating within an organisation, if you’re deploying SharePoint, but it means a limited audience. It is easier to create stuff in Google Docs if you want to publish on the Internet. And what happens as people start to publish more using Google Docs? Others start asking where they can go to get the same features, things they can’t do within Office… I’m not saying ditch integration with SharePoint, but it shouldn’t be the only option.
3. Get an online edition of Office out there for people to start using
Online services aren’t necessarily replacing traditional applications, they are creating new ways of working with information. Ways that might not come up in focus groups and product planning. Not having an online service risks being late to a game you are supposed to lead. A big win for Google Docs has been enabling people to collaborate simultaneously on spreadsheets in real-time over the Internet. In this scenario, browser (and online service) is best because all collaborators are, by default, using the same version of the software and spreadsheet. They don’t need to purchase, download or install anything to participate. But when it comes to features, people don’t want full Excel inside the browser – it would be too complicated. Equally, I’m not convinced people want to tolerate browser limitations all the time. Somebody who provides a marriage between the two environments might just be onto a winner. About that software+services strategy…
I always thought SharePoint was key to securing the future for Office. Now I’m not so sure. Too much of the current Office Live strategy appears focused on using SharePoint as the hosting platform for all things web-related to do with Office, whilst Office carries on doing what it’s always done. People are using Google Docs not (just) for a cheap spreadsheet in a web browser, but for scenarios that benefit from the spreadsheet being in that web browser. Here’s a thought. Many organisations will not use Google Docs because it is hosted on the Internet. Imagine if Google decided to add Google Docs to their Enterprise Search Appliance…
[Disclaimer: I used to work at Microsoft. I’ve got a non-disclosure agreement (NDA) with Microsoft. I saw a very early preview of an alpha version of the SharePoint bits in Office 14 (the next release) last year under NDA which means it in no way influenced this post. This little splurge came from an hour playing with a spreadsheet in Google Docs and being surprised that it did somethings better than Office 2007 and SharePoint. It’s the first time I’ve seen Office in the same light as mainframes when client/server entered the workplace. They didn’t start by replacing the core mainframe applications, rather they took over the functions that mainframes had introduced – e.g. spreadsheets, email – and were doing badly…]
Now, for a far better example of the benefits of motion charts:
[Update: 27 Apr 08] A Google Docs vs Office comparison has been posted at GoogleSystem – Google Docs Lives to Share the Words
Back in January, I wrote a blog post about using Twitter for Sales.
Interestingly, when talking to my parents about Twitter, they bought into it immediately. Their immediate thought was Twitter for Families – why didn’t everyone get on Twitter so that we can each send out updates to everyone when stuff is going on. Twitter hit a nerve that blogging, Facebook and friends all missed. I now use Twitter pretty much daily when appropriate. Often to simply post simple quotes that make a point. I’ve got a Twitter feed on my site home page which, given I can Tweet from my mobile phone, is the easiest method for dynamic updates I’ve ever implemented…
And now Michael Arrington over at TechCrunch has a great example of a company using Twitter to improve customer service – Comcast, Twitter and a Chicken. Fed up with getting nowhere with Comcast when his Internet connection died, he used Twitter to start criticising Comcast for being, well, not that great. 🙂 It turns out that one wise exec at Comcast monitors Twitters and other sources. He saw the criticism and debate that kicked off, contacted Michael and a support team turned up to fix his problem. That’s real-time customer service.
These are the sorts of benefits too many companies miss out when sticking their heads in the sand about new technology.