From Grid to Cloud ...

The end to this chapter ... Turning the page!

2011-03-29T09:56:00.000-07:00

Dear friends, the Digital Systems KTN finishes with its funding on March 31st. At this point the story of the rise and fall of Grid Computing and the emergence of Cloud Computing can be told. From the 1st of April I will shift to a new blogging platform, just for the change, and you will already find my introductory piece here.
I have been the Director of a KTN for 6 years and have had the privilege of working with a fine bunch of people from Malcolm Atkinson and Dave Berry at the National e-Science Centre at the University of Edinburgh, to Tara Kelly here at Intellect; Gillian Law at both NeSC and Intellect; and Mahesha Pandit here at Intellect. Along the way Mark Parsons covered our interests from EPCC also at the University of Edinburgh when Dave Berry left for service-orientated pastures anew. We have been grateful to many people in these organisations for their help as we learned how to deliver against the KTN vision, in many cases helping define it too!
We have also been grateful to colleagues from our industry, Dave Pearson, John Barr among many others, who have provided the intellectual horsepower, encouragement and support to allow our agenda to flourish and grow.
The last 2 years have been dominated by the KTN "optimisation" agenda driven by our paymasters the Technology Strategy Board, and we have successfully brought to life a Digital Systems KTN in the wake of the Grid Computing Now!, Cyber-Security and Location and Timing KTNs. It has been a pleasure to get to know Bob Cockshott and Tony Dyhouse and their colleagues at NPL and QinetiQ, as we have pulled together into a new KTN, in the full knowledge of the further change ahead. We are now merging with the Digital Communications KTN in the final stage of optimisation and my colleague Philip Hargrave and his employers the ICT KTN Co., will take over the administration of the grant from the TSB.
This leaves me free to concentrate on the task at hand, and you should read my new blog entry for more details on that. I will also have some more time to spend with my colleagues at Intellect, the UK High Technology Trade Association, too. I'm certainly looking forwards to spending more time on innovation and much less on administration!
As for the bottom line of our work, we publish the latest Annual Report for the Digital Systems KTN in the next few days and there'll be data aplenty to see there. But for the casual reader, we've successfully grown a new community of 1800, and rising, around our programme in the last year. Inheriting relatively few from our "legacy" communities. We've participated in more than 150 events, some of which were our own - but most of which enabled us to address new audiences. And we added 15 case studies to the portfolio making a total in excess of 60 available online, the vast majority of which describe successful adoption of distributed computing. I am also delighted to relate that there have been several notable successes in the adoption of distributed computing which I have had the privilege to personally witness in the past 6 years. Perhaps the most promising is eMediaTrack, an UK based start-up, now firmly rooted in Oxford for technology development purposes, while plying its trade in Korean Government. Korea is a nation which has stated a desire to "lead" the race to the Cloud for Government computing. I know, I was there at the event they announced it!! I was also personally involved in the development of the UK G-Cloud strategy for Government computing. This story is still being written. But the ideas are of great interest within and beyond our shores.
For those of you interested in the Grid to Cloud story. I think it is safe to say that the future of distributed computing, and services, lies firmly in the Cloud. Recent announcements of Grid services available in the Cloud have possibly made this point even more definitively. Recent announcements of cloud services available from national infrastructures also make an important point about ease of use and broad applicability of cloud based services. However, please don't be confused. Grid computing underpins the service infrastructures for all of the leading service providers. Open standards of the type being championed by OGF, are pivotal in the operation of these companies. And much of the learning from the Grid community will eventually come home to roost in Cloud services I am convinced. So with that thought I propose that we should declare victory at this point and move on to the brave new world of IT as a Service. That's my plan!

So many opportunities, so little time ...

2010-10-01T08:53:00.000-07:00

So the summer in the UK has ended, as any of you Ryder Cup enthusiasts will have noted, and before I pack up my pen to end the official working week, TGIF, I thought I must blog. I've spent a good proportion of my time this year talking about Cloud Computing in general and Government Cloud in particular. In fact, I started work with the G-Cloud Phase II team at this time last year. We produced our final reports in February and have been waiting since. Although a General Election and new Government has intervened. After 12 months we are now awaiting the results of new Government's planning to determine what happens next. What is certainly the case, is that we are all agreed - in the IT community at least - at what the opportunity is, what the challenges are and that there is no more money to be spent on doing anything. A recipe for an hiatus, I hear you say! Yes, well in ordinary times, perhaps that is so. But these are far from ordinary times. We heard yesterday that Ireland is on the brink of returning to its unenviable situation of being a poor country. The UK is not in the same boat, yet there is so much potential for us to change the way we do things, release the potential of exciting new ideas and perhaps the people to work on them. So why don't we grasp the nettle and move on? After all, it's not as if our current jobs are safe, is it? Why not reach for the stars and grasp the challenge and see what we can accomplish. After many years of ups and downs in the IT industry, I can honestly say that making do with less is preferable to not being there at all. In fact, a wonderful article in Harvard Business Review by Gary Hamel and the late CK Prahalad, "Strategy as Stretch and Leverage" pointed out that having insufficient resources will often lead to making choices and requiring innovation to get the job done. Why not capitalise on the genuine lack of resources and plan to deliver services differently? Share similar services with other organisations, outsource a large chunk of routine information processing to a SaaS vendor? Better still stop doing something which really doesn't matter or could be done elsewhere instead? After all, if the reasons for not doing so are rules written within the organisation, why not challenge the organisations own ability to manage change and do it anyway! You'll probably end up leaving anyway, why not try something new and see if it works? After all, your job is probably on the block anyway! How else would you wish to spend your life? Have a good weekend - at least!

Virtualising the Workforce

2010-07-28T02:32:00.000-07:00

I've been intrigued in recent days by the idea of procuring virtual workforces to complete projects successfully bid for by lead organisations. One or two leading organisations in the UK are toying with bringing such a service to market. The proposition is particularly useful for smaller businesses, aiming to bid for more business but wary of commitments to recruit a permanent workforce on the off-chance of winning. It works thus: A business has proven expertise to structure and deliver a service to a customer but does not have the capacity to flex to meet demand. It engages a third party with access to skilled resources to back its bid and on winning the business, the project team is deployed in the name of the contract winner to deliver the goods. Such a model relies upon clear leadership and competence of the lead organisation, proven skills and capacity from the service provider offering the skills required. But, its a realistic model for the more fluid organisational world that we are migrating towards. It offers the same sort of flexibility to scale that Virtualisation and Cloud offer, and for the skilled individual, the ability to gain valuable experience and add to a CV in times which are proving challenging for permanent employment prospects. Its a win, win, win. For those of you who think that this is a new concept, I commend Charles Handy's book Inside Organisations and particularly his work on the Clover-Leaf organisation. This is a 1980's vintage piece of work in which Handy, a management researcher, speculated on the form of future organisations and the decline of the first job to retirement employer in place to that date. Its a live topic today, see http://www.flexibility.co.uk/flexwork/contract/birkbeck.htm.

UK Government and the Green Challenge

2010-07-20T08:34:00.000-07:00

Things are moving at pace in the Cabinet Office in the area of Efficiency and Reform Group improvements. The latest being an incitement to work together with contractors to reduce energy usage by 10%. See here . This is a call to arms from Government to their network of service providers to help to accomplish the targets for improvement. This specific call is to adopt the new Energy Efficiency Code. This states that all departments should:

work with facilities management contractors to prepare plans to cut carbon by ten per cent by May 2011;
give serious consideration to private sector ideas to cut carbon emissions; and
agree to explore agreements, possibly through changes to existing contracts, which benefit government departments and private companies in their pursuit of reducing carbon emissions.

In response, contractors are asked to share their expertise with public bodies and actively identify opportunities to reduce energy emissions.

Could this be the pattern of things to come? I think so. Helping to achieve targets will move the government forwards and perhaps is the best portend of the opportunity ahead for the G-Cloud programme. Now headed by Chris Chant and Andy Tait in John Suffolk's team at the Cabinet Office.

To help with the reduction in Carbon Emissions, we have a number of tools in our armoury in the IT Industry, perhaps here is the opportunity to begin to show what they can accomplish. After all, didn't someone say that Green IT is free? I.e. we should be able fund investments to reduce energy consumption through savings on our energy bill and operating costs?

And once we accomplish improved efficiency in our own shops, we can use our capabilities to support collaboration and remove the need for travel and commuting, and thus really gain traction in saving carbon emissions. The Carbon Neutral Data Centre, a possibility?

Certainly investments such as Virtualisation and selective Technology refresh can make a positive return. See our KTN website and the Green IT group for more!

Trusting your Cloud Service Provider

2010-07-16T04:25:00.000-07:00

One of the key challenges faced by users and suppliers alike in the new world of IT as a Service - is who am I dealing with? What can I find out about them? Can I trust them to run a service for me? Will they be there tomorrow. Colleagues in the UK service provider industry have developed a proposal which might help. The Cloud Industry Forum is an initiative which seeks to allow Service Providers to declare themselves in a coherent and open fashion. So that you will know who you are dealing with, who owns/runs the company and where their facilities are. These are key elements in developing an outsourcing relationship and this initiative is a good start to the process of openness. Let's face it, we are walking into a world where we do business on a personal or a professional level, with people we have never met in places we don't know exist. You try finding out where Amazon has its Data Centres! Take a look and let the team know what you think. Next step is knowing who we are when we strike an online deal with a service provider ... but that's another story!

Making sense of a Life?

2010-06-21T03:18:00.000-07:00

Just a bit of fun this one: I encountered an imaginative way of presenting a CV listening to Shaun Frolich of MS last week. He used the Wordle tool to represent his CV. What fun? Well, I had to have a go and here you are ...

Digital Britain, Building Britain's Digital Future and G-Cloud

2010-04-07T02:30:00.000-07:00

So yesterday's announcement of a General Election to be held on May 6th in the UK was extremely well trailed and predicted by all. Even the Queen was able to 'copter back to Buck House in time to see the Prime Minister as he requested the dissolution of Parliament. Now we have just one month to survive hostilities until its done and we move on.
However in the run up to the election several things of interest to the digirati have surfaced. Firstly, the Government published its Government ICT Strategy in January and announced plans for a Government Cloud. This is a longer term strategy to enable the government with the capabilities to bring services to the web for citizens, of whom an estimated 71% are now online. The key issues are accessibility and scalability and the Cloud computing model works well to support both objectives with suitably designed applications and infrastructures. I myself managed to obtain Road Tax and a new Driving Licence on line this month. The latter being made possible by linkages to the Immigration and Passport Service. We've also seen new services offered by Ordnance Survey which take advantage of the Cloud to scale to meet demand from up to 9M users simultaneously. So there is potential.
Secondly, and probably more political was the Building Britain's Digital Future announcement on March 22nd from Number 10, the web residence of the erstwhile Prime Minister, of plans to revolutionise the delivery of all government services (ex-Health Service and Dustbins one presumes) via the Internet. This is the logical consequence of following the advice of the recent notable advisers to government Sir Tim Berners Lee and Professor Nigel Shadbolt from the University of Southampton. Both of whom have already exercised their enthusiasms for the Web through the data.gov.uk initiative. Which, by the way, if it was extended more broadly as the norm throughout government could probably yield the sort of savings the political parties are looking for in avoided Freedom of Information requests. Has anyone asked the price of that as yet? This was a bold vision and exciting prospect for those of us who have laboured over the G-Cloud project vision without much political involvement. Well worth a read, and you might expect it to pitch up during the hustings!
Which leaves us with the 3rd, and perhaps key enabler for the stool of Digital Services to the Citizen. Last night in the mad panic that is the final act of a dissolving government, the Digital Economy Bill was passing through into legislation. The problem is that the strong lobby of the Creative Industries (cf copyright owners) have included measures which allow for undemocratic, if not immoral, powers to be exercised by Internet Service Providers where illegal file sharing is deemed to be taking place, without obvious recourse to law or appeal. The key remedies include removing internet access from locations deemed to have transgressed - by the copyright owner, one presumes. The unintended consequence may be that the current proliferation of wireless hot spots across the nation may be inhibited by those service operators wary of being caught in the middle of disputes. An half considered plan with far reaching consequences would be a tragic conclusion for a Government so keen to bring the UK into the 21st Century and certainly a wasted opportunity to move forwards. What price 2MB to the doorstep and a £.50 a month levy on wireline access?

Perceptions on Security and Cloud

2010-02-12T09:26:00.000-08:00

So, I attended the Westminster eForum this week to join a panel in their Cloud Computing event, graced by none other than Reuven Cohen, yes he of the Cloud Camp! While my panel session was all about the benefits of the Cloud to organisations in Public and Private Sector, much rehearsed and not to be repeated here, I was more taken by the discussion in the following panel which focused on Cloud Security. In particular, discussion centred around the differences between security vulnerabilities in the Cloud, versus security vulnerabilities in the tradition context. I have previously observed that the Cloud might give at least one genuine advantage over the corporate firewall, it will be hard, if not impossible to physically steal an asset when you don't know where it is! Hard to break in to a data centre when you don't know which one has the data!

But seriously, the real difference is what happens when you remove the illusory security of the corporate firewall, remember your biggest risk comes from within, and therefore users will have to secure applications and their data in isolation from the environment in which they are executed. Isn't this much like the discussion of deperimterisation which was raging a few years ago, operating securely across the internet for decentralised organisations? What does it take to operate securely in the Cloud? It seems that the way to think about this is to imagine deploying a service in which each component is secured, from virtual machines which are pre-configured with anti-malware, robust, multi-factor identity and authentication procedures, information which is encrypted during transmission and storage and processes which are self-cleaning on completion or abortion of execution.

It seems to me that this is common sense and a good example of systems engineering at work. After all, does the opposite hold true? I.e. that you can leave information lying around anywhere within the firewall because only employees (and contractors) can access them via the intranet. I think not. I guess that there is no substitute for some hard work!

Government Cloud and Open Standards

2010-01-29T10:09:00.000-08:00

On Wednesday, January 27th 2010, the UK Government announced its new ICT Strategy for meeting the requirements of the citizen, both in delivering services for the next decade, as well as meeting the requirements to reduce public spending through consolidation and sharing services and capacity. At the heart of the strategy is a commitment to implement a Government Cloud or G-Cloud comprising a shared infrastructure and an Applications Store with which to gain access to IT services, whether automated or human, for civil servants. The vision is to encourage more effective use of resources, starting with a pilot implementation of Infrastructure as a Service, followed by Data Centre consolidation and the development and deployment of shared services across government. This can result in a substantial reduction of costs, through removing duplication of services and resources, a reduction in energy costs and a consequent reduction in carbon footprint. This is by no means the only example of a government initiative on Cloud adoption, the US led the way starting a year ago with its Federal Cloud Computing Initiative, which comprises a similar drive to offer Infrastructure as a Service and a Storefront to allow access to shared services. The Canadian, Australian and Japanese Governments are also looking at their own offerings too.
One of the key drivers in the G-Cloud and FCCI initiatives is the development and adoption of open standards for cloud service providers encouraging the possibility of interoperability. Bob Marcus and Craig Lee, OGF, are collaborating with others, see cloud-standards.org, to help bring focus on this important process. This will be discussed at two forthcoming events in Europe:- 1) CloudScape II being held in Brussels on February 22nd and 23rd; and 2) OGF 28 being held in Munich on March 15th to 19th. There is a lot of work to do and recent developments within the OGF with the OCCI group are beginning to surface some key areas of understanding for promoting utility computing. All in all, I believe we are witnessing a significant shift towards shared infrastuctures supporting shared services and, somewhat unusually, the governments of the world are well equipped to motivate some of this innovation and thus improve services while reducing costs. Bring it on!

The on ramp to the Cloud

2009-05-26T06:25:00.000-07:00

So where are we on the way to the Cloud? In spite of substantial amounts of noise from media, un-conference and un-workshop activities, the reality is that true commercial use of the Cloud has been limited. It is widely recognised that the leader in the space is Amazon Web Services and that de facto, the EC2, S3 and plethora of services offered by Amazon offer the current benchmark for competitors to approach. Their announcement of new, dynamic building blocks or tools, to allow the scalability of service delivery using Cloud shifts the agenda another step forwards. See http://fountnhead.blogspot.com/2009/05/new-aws-enable-real-elastic-clouds.html for a useful summary of the changes.

Industry rumour has been working overtime on Microsoft's Azure offering and the specific services being offered to their enterprise user base, where the flexibility to offer applications as a service and/or supplemented by cloud based services is proving an attractive prospect. Equally, Google is flexing its considerable muscles to offer its Google Apps and Docs services and with possibly the largest global infrastructure of any vendor, they are well placed to extend their services in a similar vein.

However, the word on the street - or in the Data Centre, is that company data will not be disseminated outside the firewall - over the dead body of the CIO. In fact, given that most shops have ample processing power available and rapidly escalating storage needs, the need to acquire new resources outside the firewall is actually quite low. Enter the discussion about Private Cloud. One of my roles is in the global distributed standards organisation, Open Grid Forum (OGF). In this group we are interested in identifying the building blocks of open standards which allow the end users to assure themselves of interoperation and consistency across their chosen software vendors. As the Enterprise lead for OGF, I am interested in understanding how IT users would really like to incorporate IT Services and Cloud infrastructures into their provisioning and infrastructure management tools. To that end we are running a workshop here at OGF 26 in Chapel Hill this week, which aims to start the process of understanding user requirements and use cases to flow into the OGF Open Cloud Computing Interface (OCCI) standards activity announced in April. OCCI is certainly gaining some traction in the space,a lthough the industry leaders are not yet convinced of the benefits of open standards applied to their services - that's the challenge for the prospective users of these services to argue that it is in the best interests of their vendors to comply with their needs. However, it is early days.

Multicore - challenges on several fronts!

2009-02-27T08:05:00.000-08:00

So it was my privilege to host a GCN! Webinar on Multicore yesterday, Thursday, February 26th. See www.gridcomputingnow.org for more. I have been looking at this area for a while, wondering what steps have been taken to ensure that the new cores being placed on your desktop will be utilised in an efficient manner. It turns out that this is a questionable expectation.

The key points made in the webinar, which featured colleagues Francis Wray of Concertant LLP and David Henty, of University of Edinburgh, were that:-

the use of parallel programming techniques are required to take advantage of multicore and other parallel architectures.
these techniques are essentially of two variants: shared memory or message passing.
In the former, programme components access a single pool of memory, handling all the administration and controls associated themselves.
In the latter, information is distributed through the infrastructure and operated upon in parallel.
For higher performance, message passing is deemed superior.

Now how many readers of this blog, remember these facts from their training? How many have used them in anger?

Frannie, frightened the life out of me when he said the thing to do was to build an application and then see how well it runs in a multicore environment. Have we really not got anything better to show for our 30 years of development than traditional hand crafting?

It certainly seems as though there are some commonly accepted language extensions which allow for the definition of shared data structures. There are also some common sense guidelines (suspend disbelief on this) for parallel programming models. And some tools, such as compilers and post coding analysis tools, which can help.

But bottom line, it looks as though you are on your own if you want to develop applications targeted at multicore environments. And, don't expect any help from the O/S if you simply want to get the best out of your dual core laptop. It would seem that the plethora of "services" that are running on your pc in background today, will preempt any "threads" which you fire up in parallel, thus hobbling the performance of your application. And that's assuming that you can figure out how to get the thing to work!

Bottom line, have we been sold a pup? The idea that Moore's Law continues in the face of rising heat and power requirements through the deployment of multicore is in my mind extremely dubious. It might be the case if the compiler/operating system helped create and allocate threads effectively. And that the O/S allocated the work to processors in a sensible way. But if that is not the case, are we simply going to leave the other processors on the die unused while the only one we can access melts itself with exhaustion.

Intel is right to moan that the software industry is not keeping up. But what steps has it taken to raise this issue and ensure the right tools are available? Please let me know. By the way, Sandia National Labs have recently published a study which questions the multicore strategy beyond a few processors. It seems as though the memory contention and communications challenges that are presented when large numbers of cores are present tend to defeat the strategy. See http://www.spectrum.ieee.org/nov08/6912 for more.

And I thought that merely using the cores as servers for virtualisation would do the trick!!

Taking it to the Cloud!

2008-12-12T10:21:00.000-08:00

So a large amount of vapour has passed under the bridge since my last post. As is always the case, good intentions are always overtaken by necessity, especially on Friday night! However, this week I am faced with a lengthy wait here at Heathrow's T5, not all the fault of the incumbent airline, but nevertheless, I am sitting here with some time to spare and so here goes!

In recent weeks I have attended several conferences, including ICT 2008, the European Commission's big research and development event in Lyon, City#Grid in London's Canary Wharf, one of the premier computing conferences for Financial Services, and other conferences more closely related to my day job in Knowledge Transfer, Innovate '08; IDC's Green IT, etc.. A couple of things which have been dominating recent discussions in the industry at all these venues. The Cloud! It seems that the developments in IT as a Service are beginning to really capture people's attention, there are challenges, but the recent economic downturn is again turning people's heads towards cost cutting activities and what sounds better than replacing agening equipment with shiny new objects in ths cloud? Well it's not as simple as that, but wait there is help at hand!

On January 14th and 15th, one of the projects with which I am associated, OGF-Europe, is holding an event called Cloudscape, A Workshop Exploring Cloud Computing and its Impact on Enterprise IT , in Brussels. This is a programme which has been put together by our Industry Expert Group, to bring focus on the developments in the atmosphere and allow for some genuine engagement between practitioners from both sides of the cloud. Chaired by Microsoft's Fabrizio Gagliardi, the conference will feature speakers such as Paul Strong from e-Bay and Kyriakos Baxevanidis, Acting Head of Unit Géant & eInfrastructures unit European Commission. Among several prestigious contributors. My colleagues from the Open Grid Forum (OGF), a distributed computing standards community, another organisation with which I now have strong links, will also be in town and we hope for an healthy exchange of ideas and experience with which to start the new year. If you are interested please register for the event here.

And for those wishing to bundle their trips together, the SSOKOU '09 conference is also being held in Brussels immediately preceding the Cloudscape event. This event is all about the future of services and knowledge utilities. Its being run by the European Community for Software and Services. Register here. Perhaps I'll see you there?

Ian

Grid Computing Now! So did we do the job?

2008-08-25T08:36:00.000-07:00

There's a debate raging in the Google Cloud Computing group about what are the key lessons to be learned from Grid Computing in bringing the notion of Cloud Computing to the market place. It sort of runs like this: "we don't want to make the same mistakes as Grid!" to which the grid afficionados involved will respond with "what do you mean mistakes? Grid has been successful and Cloud is but an offshoot of Grid!" Those of you with any lengthy experience of the computing geek mind will know what happens next. A text version of the pantomime comedy line "Oh no you didn't"; "Oh yes you did!" delightfully summarised in the immortal Monty Python Argument sketch as "a simple act of gainsaying".

I have made my living for the past 3 and an half years from the proposition that "grid computing is good for us and we should all adopt it!", stemming from the work done in the UK e-Science programme over the past 10 years. Sitting here in 2008, it doesn't feel a bad place to be. Our collective view is that the best of the ideas from grid computing middleware, regarded as configuration, resource allocation, scheduling and job management, seemed to have been incorporated into a set of useful tools for system management and virtualisation.

Thus we have distributed computing infrastructures, often heterogeneous, and useful means for increasing utilisation in the data centre. Is this Grid, well for the purists, no, the infrastructures are usually proprietary and therefore not Grid, but for the pragmatists - of which readers will observe I am one, the answer is yes, this seems to have the right effect and its not bad for a start!!

What do you think the progress with Grid adoption has been over the past 3 years? Can you point to successful examples? Or proof points that its not happening? I'll be interested to know.

Meantime, at the world watching paint dry championships ...

Green IT, Carbon Economics, Missing the point?

2008-08-01T01:45:00.000-07:00

So just returned from a brief break away in the Deep South of England where I indulged in the usual process of theatre, Stratford and RSC - Respect!; the Guildford Cricket Festival, Surrey - you've lost your mojo! and meeting friends and family, the Thai Terrace in Guildford, what a glorious oasis in suburbia! During my break I studiously ignored my BlackBerry - a habit fostered with the introduction of e-mail at HP in 1982 - and only occasionally kept up with the news. However, on return I came across the announcement of a government "green" initiative which hopes to achieve carbon neutraility by 2020.

My colleague, Jerry Fishenden at MS, has previously pointed out that there is an huge opportunity for IT to contribute positively to the reduction of carbon emissions through the effective deployment of services and reduction in waste that can be accomplished using applications. Imagine, full trucks going both ways on the M6; your local white van man making only one tour of your district to deliver web-ordered goods; your choice of transport mode influenced by early warning of traffic delays; etc..

In addition, the news this morning featured shock/horror stories of the fact that the UK carbon footprint may well have grown since 1990 once the total lifecycle of products and services consumed is taken into account, and you'll get a sense of where I am standing on this issue.

I believe that we have an huge opportunity in ICT to revolutionise the way we do work and deliver products and services and that by concentrating on the real challenge of delivering this innovation we will bring into the correct perspective the amount of energy that is expended on ICT equipment. IT is not the problem, it is a significant part of the solution.

Yes, we need to bring our house in order, improving utilisation, reducing power consumption and improving cooling and other energy sinks in our data centres and desktops. But ultimately this is the side show. After all an unnecessary journey in a car is still unnecessary, whether or not the car is an hybrid running at 100mpg or a Ferrari running at 15mpg.

As an industry we need to act as one to bring real innovation and process to this. Let's learn from the leaders in the industry, many of whom are well advanced on both fronts, innovating in the delivery of services as well as the costs of delivery. I really think that the government should look at the benefits accruing to the NHS with the National Programme for IT, and then look again at the state of IT in central government and prepare for significant change. Yes it might take an investment of £10Bn to move forwards, but if planned and executed correctly the efficiency savings alone might pay for the benefits, let alone the potential of removing thousands of commuters from the daily treadmill by opening up new working practices. Its not just about switching equipment off!

Grid Computing for a Greener Planet

2008-07-04T04:40:00.000-07:00

I'll take a break from my usual stream of consciousness blogging, at least I'm claiming that!, to let you know that Grid Computing Now! has launched a new competition for 2008. Entitled "Grid Computing for a Greener Planet" we're jumping on the Green IT bandwagon - although we've been there a while now - see previous blogs ;o) This competition is aimed at students and IT professionals, to generate some new ideas about applying distributed computing to the task of reducing carbon emissions.

Now it's tempting to concentrate on the consumption of IT equipment and its venues in this context. This is indeed important, but the real dividend lies in the use of IT systems to optimise our use of energy and consequent carbon emissions in every field. Consider the way a supermarket can optimise the use of its warehousing, distribution, retail and home delivery activities through efficient management and control. Consider how the optimum use of transport facilities can minimise time spent wasting fuel in traffic queues, or even avoided using IT and communications facilities such as video conferencing.

I'd like to think that those of us who work, or plan to work with IT can find new and exciting ways to change the way we live and reduce our carbon footprint. If you agree, then this competition is your vehicle to help change the world! See www.gridcomputingnow.org for more!

Sustainability: So how do we break the mould?

2008-06-17T08:34:00.000-07:00

Sitting here at the BT Centre in London where I am attending the ITU conference on ICTs and Climate Change, see http://www.itu.int/themes/climate/. The conference is focused on the challenge of climate change and the necessary steps to be taken to achieve a balance between man made emissions and a level playing field for developed and developing worlds looking forwards. Most notable during the day has been the sequence of contributions from the hosts as they have presented their results to date. A very creditable and real reduction in carbon footprint. See the presentations, via video and powerpoint, available from the website.

However, that's not the purpose of this post. I also attended the SIIA conference in Amsterdam last week (no jibes about carbon footprint this week please!). I was most impressed by the rapid evolution of SaaS and the very practical steps that people are taking to offer new services and capabilities on-line for consumption via the simple browser. Here was a genuine low-carbon conference dominated by the plans and capacities of an industry structuring itself around Infrastructure as a Service, Software as a Service, Information as a Service. The global champion in the room was Amazon Web Services, again through the person of Werner Vogels, see my previous post. But the buzz and excitement amongst attendees was palpable around the delivery of business services through these mediums. There was even a session focused on Security which was far less frightening than removing an orange folder to take your seat on a train at Waterloo Station! General consensus, this is real - and its changing the game!

So, to my point, we seem to have a created by default a product development cycle and roadmap in the computer industry which assumes: 1) that the next box will be more powerful, have more capacity and probably expend more carbon resources as a result. 2) we will fill this new box with a larger amount of software than we could ever imagine using, and thus negate the benefit of increased power and capacity and 3) will be persuaded to exchange this barely utilised resource in 3 years time because of a step function increase in processing power/storage capacity and an impressively colourful lid.

When you consider that data presented at the conference indicated that desktop pc's consume about the same energy to operate over 3 years as to manufacture initially and then the disposal will consume more, irrespective of potential/process for recycling, the question "WHY?" pops into my head. When you combine this question with the apparent potential of XaaS and perhaps a simple wireless connected browser. I begin to wonder why we are continuing on this treadmill of a roadmap and all of the inerent costs implied.

In one panel session at the ITU conference, we heard from an EC officer about increasing regulation and a trade association representative about allowing market forces to take their effect. At present, we as users, are running fast to continue to buy the newest, more powerful and therefore likely most expensive option to solving their business problem. If you believe in market forces, then I am afraid it is those of us who use systems who will need to drive this change forwards. Who's with me?

Barcelona, Grids and the Cloud

2008-06-03T07:37:00.000-07:00

This week is OGF 23 in Barcelona, a global grid community gathering in support of the agenda of grid standards and adoption in the scientific and business communities. The event is also shared with BEinGrid, an European initiative to demonstrate business relevance of grid, and the launch of OGF-EU, an European FP7 project aimed at championing the standards cause and European contributions to the development of grid. I'm pleased to say that my employer is supporting OGF-EU as well as my continued stewardship of the Grid Computing Now! Knowledge Transfer Network.

So what's the buzz, we're two days into the event and a pattern has emerged, right? Well, yes, it has. Simultaenously, we are told that Grid is not being adopted as expected and that Cloud is the answer. We also had an interesting debate following Werner Vogels excellent keynote speech, he's Amazon's CTO, about whether the "cloud" is in the grid or the grid is in the "cloud"!

There are grains of truth in all of these propositions:-

Firstly, grid is not likely to be widespread in its implementation as a technology. Its proposition is that you are able to distribute your applications to operate in parallel in an heterogeneous compute environment, within and possibly, without your organisation boundaries. This of course requires some form of restructuring of applications previously written with specific hardware configurations in mind. Not a likely course of action for most understaffed (in capability and numbers) IT departments. Without doing this the application cannot take advantage of a grid infrastructure. In addition, configuring grid infrastructures to support transaction processing is in its early days, outside those experts in the major application suite suppliers and leading web service providers, all of whom are building their applications to distributed infrastructure architectures. So, the likely range of adopters is limited.

Secondly, cloud is the next big thing and a sexy concept to discuss. The reality is that the leaders in the cloud service provision industry, which certainly includes but is not limited to Amazon with its Web Services, are simply dipping their toes in the water with large scale commodity infrastructures leveraged from their understanding of the their own e-commerce business models. Remember that the likes of Amazon and e-Bay already operate huge e-commerce infrastructures and literally service millions of suppliers, drawn by the lure of customers! However, looking forwards, the scale of infrastructures being built by these leaders and Google and Microsoft and ... is truly gigantic, often environmentally efficient - in a way that those with data centres in the centre of big cities really can't compete with, and they have the capacity to support many customers before they start to sweat on further investment. The rub is that they have invested in low technology commodity hardware and high technology competitive management practices to deliver resilience and efficient operation - all in the name of keeping the cost of services low - and much lower than the owned data centre!

But you say, doesn't security floor this proposition? Yes it does, but not the way you may think. Its governmental and industrial regulation which is providing obstacles to storage of data on non-dmoestic infrastructures; or an audit trail which forces the trading company to invest in masses of expensive, redundant on-line storage. Perhaps there'll be a white knight government or sector who will look again at these restrictions and the added costs they incur.

Finally, to the Grid in the Cloud or Cloud in the Grid debate? Firstly, who cares? It is a trusim that grid computing technologies are at the heart of all of the new "cloud" infrastructures. Its also true that people can implement grid, which means installing and running middleware in cloud instances to form a virtual grid. So you tell me which came first, the grid or the cloud!

The bottom line of all this? What price the owned data centre of the future. Is it possible that wholesale change will impact the enterprise data centre as CFOs struggle to reconcile a cost m odel which is 10x that of the leading cloud service providers? Who else will be there? Well, the telco's are looking at network service provision with an eager eye. The Big tin vendors are increasingly being pushed down the path of adopting large scale customer infrastructures. Where might they make their profit? In the Cloud!

Software Licensing in the Virtual Age

2008-03-19T10:12:00.000-07:00

A lot has been written about the irresistible march of Virtualisation in the data centre and now on the desktop. There is no doubt that for many users of IT, and those that pay the bills, the ability to increase utilisation through the use of virtual machines embodying increased capacity to serve, is a boon to business. Leading to lower running costs and accelerating the ability of the IT infrastructure to react to business demand.

However, it is a rare beast that has only benefits and no costs or disadvantages that accompany. Nor is it the case in this instance. For we have the equivalent of a new fire burning away in the IT shop and virtualisation is simply adding more fuel to the conflagration. The challenge is that we already struggled to keep track of software licenses, and virtualisation is systematically breaking the bond between user and software and between hardware and software, such that it is very difficult now to see where and how many instances of software are executing.

Not a problem for your business, I hear you say. Well, you might consider the following questions before dismissing out of hand:-
1) How many licenses do you have in hand for your organisation? Is the agreement made based on number of employees? Hardware on which the software is installed? Some form of licensing mechanism which allows for licenses to be granted dynamically?
2) Are all instances of software run on the same premises, or spread across sites in the UK, and/or beyond?
3) Do you use open-source software in the same manner?
4) Do you use multi-core processors in the data centre? On the desktop?

Unless you are very well in control, I would suggest that most of you would find it hard to be equivocal in the answers of any of these questions, except perhaps for Yes to questions 3 and 4. In which case, you almost certainly are flouting your software licensing agreements in the virtualised infrastructures you are creating.

This is an area of concern which has been troubling many organisations at the leading edge of Grid and Virtualisation implementations for some time - and now its going mainstream. I would suggest that you check the license terms and conditions for your most common applications and then check to see the audit data and trail that you have in place for their management. If you are struggling to keep up with developments (and by the way, your developers now have new found freedoms to introduce servers at a drop of a hat!), join us for the Grid Computing Now! webinar on Thursday, April 10th at 2pm. You can register via our website http://www.gridcomputingnow.org/ and there is no fee.

Speaking will be colleagues from Capgemini, talking about the steps you can take to characterise your usage ahead of negotiation with your supplier; Microsoft, a leader in innovating in the field of software licensing of products; and Scalable Solutions, a supplier of audit and management software for the enterprise. It'll be fun and all participants will have the chance to ask questions during the event and afterwards at an online conference session.

Look forwards to seeing you there!

Ian

Autumn - the conference season

2007-11-16T03:07:00.001-08:00

Its been a while since my last post, but there's been several good reasons for that. Most notably, our funding at Grid Computing Now! has been extended a year, which allows us to focus on two key agenda items of interest in Government and the Public Sector. The first is Green IT, and the role that a disctributed computing infrastructure can play in increasing efficiency and reducing costs and emission in the data centre. The second topic is Software Licensing, especially in this age of virtualisation. The notion that a software instance can be tied to any given piece of hardware is becoming increasingly comical. So what's the alternative?

On each of these topics we will be running programmes aimed at educating, stimulating debate in decision making circles and providing the tools and information required to facilitate change. In fact we've made a good start with the Green IT agenda with an excellent webinar on October 25th, this is accessible here, my colleague Dave Berry, the GCN! Technical Lead and Deputy Director of the National e-Science Centre in Edinburgh has added his thoughts here. Beyond this initial step, we're looking to create a community of interested members and will run some conferencing type discussion events during the year, these will feature some of the leading edge work underway at several of our members and give folks the chance to ask questions directly.

Similarly, we will follow the same approach in the Software Licensing thread of activity, although the challenges here are a little more profound. Most notably we have a challenge in breaking the industry mould in which there is a naive linking of software license with hardware on which it is installed in favour of some new models for tracking and paying for usage in a distributed infrastructure, which may span organisational and even geographical boundaries. First step in this direction is being taken with the Federation against Software Theft (FAST) who are about to lead a dialogue with their corporate user members and then with the software publishers and asset management vendors. We are planning several events with them in the New Year as the industry gets to grip with a new situation. Again, we plan to create a special interest group around this subject. I know that there are many of you out there with some frustration on this topic. Please keep an eye on the GCN! website for updates on this topic.

So the team will remain intact with funding aimed at addressing these challenges and continuing the work we are doing in the Intelligent Transport Systems space and elsewhere with the Grid API proposal for Financial Services and our general Events, Case Studies and Roadmapping activities, nb we've just published our first edition of the GCN! Roadmap our next workshop on this topic will be in May 2008.

One new capability we will add in the new year is that of providing some technical facilitation of engagements on the adoption of grid computing. We are developing a network of experts in the field from industry and academia who can help illustrate the potential of grid computing and create a context in which a prospective adopter can engage with industry suppliers. Do let me know if you are interested in this proposition as part of your business activities.

So what else has been going on? Well, apart from the recent Webinar, we met loads of good folk at the 2007 All Hands Meeting in Nottingham where we ran a session on commercialisation with the help of Jim Austing from Cybula and University of York; and Yike Gui, InforSense Ltd. and Imperial College London. It was great fun to listen to their "war stories" associated with getting businesses started from research and the challenges of maintaining the balance between commerce and research activities. See the programme here this session was on Tuesday evening.

We also contributed to another successful Innovate 2007 conference in Westminster. This annual showcase for technology innovation featured as speakers, John Denham, the new minister for Department of Innovation, Universities and Skills (DIUS) and Iain Gray, the new Chief Executive of the Technology Strategy Board, our paymaster in government. The TSB has inherited the responsibilities of the Innovation Programme hosted within the Department of Trade and Industry, which was restructured into DIUS and the Department of Business and Enterprise Regulatory Reform (BERR). The TSB is to be funded to the tune of £1Bn over the 3 years in the forthcoming spending round as per the recommendation of the Sainsbury Report on Innovation published by the outgoing Science Minister, Lord Sainsbury of Turville in October. For those that are interested, the KTNs represent a minor share of this investment, but I like to think that we make a difference. At Innovate 2007 the new Digital Communications KTN was announced and they are now looking for a leader to bring their ambitious programme to life. Keep your eyes peeled for their announcement.

And judging by the google rank for the grid computing now website, we do make a difference to some of you ;o) Thanks for your support over the past 3 years, let's see if we can build on this success to make a real difference in the next year. And then move on to the next big thing!!

Finally, our friend William Knight had a nice piece published in the Guardian this week, in which he discussed the use of Grid Computing in the oil and gas industry. Its worth a read!

August and the world is becoming Greener!

2007-08-10T05:35:00.000-07:00

I write this blog entry from my workplace in Edinburgh at the start of the 2007 International Festival and in the midst of the Festival Fringe which accompanies this event. Edinburgh becomes a city with twice the population at this time of year, the Scottish schools go back in a week or so, and so everyone is in town. The International Festival is holding its own on an increasingly crowded international stage, where there seem to be non-stop Festivals held during the summer months. But Edinburgh has one chief advantage over many, it's a beautiful city with much of historic relevance to see and sufficiently different to be noticeable in this increasingly homogeneous world. A great backdrop for the many ingenious ways people find of having fun.

But why, I hear you ask, is this relevant? No particularly good reason, other than I fought my way through the crowds at lunchtime to run an errand and just enjoyed the fact that there were so many here already, Edinburgh's Festivals (for they include the Book; Film and just finished Jazz instances) run for the whole month of August, and the fact that it was warm outside!

And that brought me back to Green IT. In the past two months, I have attended the EC's 2nd meeting aimed at developing a Code of Conduct for Data Centres; listened to a particularly well articulated argument for change in the Data Centre from BCS colleague, Liam Newcombe, and enjoyed participating in the closing panel session at Information Age's 2007 Future of the Data Centre Conference, a sort of unofficial end to the London conference season before the hols. The strong thread running through these events was the increasing concern at power usage (abusage) in the Data Centre, which when coupled with the historic poor utilisation of compute resources makes for depressing reading. It may well be that our industry is responsible for generating the same level of carbon emissions as aviation, but at least the planes carry more than an average 15% of customers!

This is a story which is going to run for a while, if you haven't already read it elsewhere, you heard it here first! The Green Grid is a supplier led consortium which is looking to bring some order and understanding to the picture, they've recently published some measures for data centre productivity, rather more related to computing function and power consumption, than delivered value. But, hey, I am not sure that we know how to measure the latter anyway.

We've also seen a rash of collapsing information infrastructures being announced, collapsing in the way that matter falls into itself towards a black hole. HP and IBM have announced eye watering initiatives to massively reduce their business support infrastructures, the former using Virtualisation and standardisation; the latter a combination of this and the resurrection of the mainframe. You can guarantee that they'll be round to help you with your plans soon.

This approach certainly seems worthwhile and if that means we can switch off older, less efficient systems, then all to the good. If it also means that we can get more productivity from our servers then even better. But there is one problem that is not so easy to deal with, and I am not talking about finding the owner of a system in the corner which has been running for years but nobody remembers what it does. The problem that does concern me is how to migrate legacy applications into virtualised/distributed environments.

If you are one of the many users of mainstream commercial application bundles from MS, Oracle, SAP, SAS, and so on, you can relax, in that these organisations are running fast to provide "service enabled" application stacks which will help in the process of migration, isolating themselves from specific hardware implementation and revealing themselves as fashionable web services. But what about the myriads of other legacy applications which are bound to a physical hardware infrastructure and may not be wedded to an abstract, virtualised infrastructure? Where do you get help for that?

There's no ready answer, other than to say your best bet is simple virtualisation at the server level and hope that the VM is compatible with your application software. There is some room for optimism with VMWare and Xen, but it's almost certainly a suck it and see equation! Still, the good news is that VM overhead is typically in the single percentage realm and process performance advances are typically in the 10-20% for each generation, so you should win, you should be able to turn off some old equipment and run those apps alongside other, newer stuff - yielding an impressive advance in productivity all at the same time!

Of course, the proper thing to do is invest in developing an infrastructure, a service-oriented architecture and assemble your applications from web services. But that's another story and one which you almost certainly don't have the time, or the budget for today. Better get those operational costs down first and fight off the Financial Director's eagle eyes when prospective savings emerge. You'll need that money to do the job properly! Good luck!!

Simulation and Modelling - New Opportunities?

2007-07-18T03:34:00.000-07:00

In recent months we've seen a major step function made in the financial services industry as grid computing is applied in a widespread fashion for the purposes of providing scalable resources for pricing, risk and modelling activities. The proposition is that the provision of a large scale distributed computing infrastructure is beginning to power new models of business to new heights of performance. Perhaps the most compelling example is in the use of Algorithmic trading models, where automated trading is taking the place of human intervention, based on the design of models for watching and reacting to market developments. The New Scientist had an excellent overview of this in its May 30 Issue . The key question is where else do we have opportunities for such innovation, changing the way in which work is done?

Well there are several areas of industry and government where simulation and modelling can break the mould, introducing innovation in the way in which work gets done. One such example is the airplane design industry where new composite materials, such as those introduced in the Boeing 787 Dreamliner, have the potential to revolutionise the construction of airframes. Similarly, the design of cars of all sorts in the automotive industry are being accelerated through the design of platform technologies and the rapid ability to generate variants to address specific market niches.

A recent meeting with the Transport Modelling industry, the potential for grid computing to revolutionise the planning of transport systems became apparent to those present. There are opportunities to speed up the single thread software applications currently in use through multiple execution in virtualised environments; there is potential to capitalise on simulation techniques used in the microsimulation models now promoted by companies such as SIAS who provide simulations of human behaviour as part of their modelling solution. This allows for the range of human responses to the same stimuli which we encounter in our journeys and thus a richer modelling result. There are folks who need to deliver traffic activity data into the scheduling and traffic adviser industries upon which businesses and individuals increasingly rely to make their journeys more efficient and/or arrive at their expected destination!

Finally, there are the big-picture transport planners from the Department for Transport's national road and freight models to Transport for London, National Rail network and London Underground, all of whom are building infrastructures to support our lives in the 21st Century. Much of the work they are doing is focused on one particular aspect; and many of the approaches adopted have their roots in historical approaches, scaled up for the challenges of today. The consequence of this are limited modelling processes which are lengthy in execution, think of days, operating on relatively narrow, independent data sets without recourse to the enrichment of data that is possible in the Google Earth/Mash-Up worlds of today.

For those of you in the science community you may recognise this pattern, its where we were 5 years ago before the Grid and e-Science community got to grips with the equivalent problems in Astronomy; Biology; Chemistry and other fields of exploration. There is some promising evidence of the potential for federating data, a project is underway at Cambridge University; and there are some interesting new ideas percolating in the area of simulation and modelling which can: a) show more graphically the patterns of activity in our transport network; b) provide better tools for reasoning about planning choices and c) provide a more compelling infrastructure for implementation. It's a brave new world out there. How can we use it to change the game to dramatically improve the way we do our work?

Roadmapping and the future ...

2007-06-05T06:23:00.000-07:00

Its been a very busy month for Grid Computing Now! after the excitement and notable success of Grids Mean Business at OGF 20, see previous post, we've continued with a new Webinar on the Semantic Web, which shows significant promise in enabling the web to be more sensibly accessed. Certainly there is a growing opinion among IT leaders that the semantic layer is essential if we are to provide meaningful searches in the future. The idea that context and terminology could be made meaningful is an attractive extension to the web metaphor.

I was invited to contribute to IT Weeks most recent Green IT webinar during the month. It's just been posted on their website and in it you'll find more evidence for adopting the newer computing technologies of virtualisation and grid middleware, to help in improving efficiency in the data centre. There's also some practical help on how to get your own Green Strategy moving.

Just after this, I attended an EC FP6 CHALLENGERS project workshop in Spain. Nice work if you can get it! At this meeting several members of the Grid Computing Now! community came face to face with colleagues from the EC grid computing fraternity. This made for an exciting and energetic exchange of views about what grid is and is not; what todays best practices mean in terms of technologies and business processes; and a long list of types of Grid that you can identify if you look hard enough. I hope that this meeting was successful in bringing some genuine industry contribution and insight back into the broader scientific community. The report will be published during the summer and further meetings will be held in the autumn and winter.

Finally, Grid Computing Now! held its own Roadmapping event in London last week, May 30th. This was attended by an invited audience aimed at identifying some opportunities for IT to contribute further to the evolution of the Knowledge Economy. A lively day was the result with a wealth of interesting output generated, which will keep us busy for some time as we refine our plans for GCN 2.0! Thanks to all who participated! We'll be making the results and subsequent roadmaps available on the website early in the Autumn. In the meantime if you have an opinion about the future of grid computing; service orientation and web services, I'd be interested to hear.

I can't help but be struck by the contrast between the approaches adopted in the planning of the last two activities mentioned. One sought to validate an existing set of principles and thoughts; the other sought to generate new insight and ideas from the existing situation. Both promised to set exciting new directions for society in the future. The outcomes from each meeting were quite different. Time will tell!!

Grids Mean Business @ OGF 20, Manchester

2007-05-18T02:38:00.000-07:00

Well, what a week it was. The OGF 20 meeting in Manchester last week created a new indoor/outdoor world record for attendance of more than 900 people, beating the previous record held by Edinburgh for OGF 5 early in the century. There were some real highlights for me in the Grids Mean Business Track, details of the programme can be found here
The conference opened on Tuesday with an excellent presentation from Tony Hey, Corporate VP of Technical Computing, Microsoft on the potential for web services to simplify demands on the underlying infrastructure and a call to action to take the user perspective of the requirements, top down, rather than invent the all encompassing infrastructure bottom up. In reality, the business agenda is being driven this way with people answering the challenges they face day to day using the available tools and technologies without much interest in standards activity. The only area where standards are particularly of interest right not is in the area of API's for middleware, where the current vendors have a lockout situation which is not in the long term interests of their customers.
The Grids Mean Business track (GMB) opened with Paul Strong of e-Bay and his usual tour de force, describing what it takes to deliver a business infrastructure serving 233 million users; 6 million product additions a day each of which is touched on average 6 times during their lifespan. He also described in some detail the processes that e-Bay use to roll their infrastructure on a two weekly basis, and I was particularly struck by their data management strategies which, apart from managing the huge scale of data employed, enable their indexes to be updated within a few seconds on a global basis. Paul did us an huge favour ahead of the event by being interviewed quite widely by the industry press so you will find plenty of sources of his business through a Google search.
The GMB sessions developed the themes of Paths for Adoption; Grid Markets and Service Delivery; Scaling Up; Collaboration and a couple of workshops on Security and Software Licensing. These sessions were all very well attended and featured many of the people who have contributed to the initial success of the Grid Computing Now Knowledge Transfer Networks.
But there were some very special contributions in the sessions on Grid Markets and on Scaling up. Chris Swann, Credit Suisse, talked compellingly about the potential for a market utility for computing capacity, his model provides for a trading Buy/Offer model, based on clearer understanding of resource requirements and scheduling set against the demand on capacity. Andrew Dolan of CitiGroup talked about the evolution of the Citi grid network and the practical challenges of implementing or evolving grid infrastructures. What's common here is the increasing scale and acceptance at all levels in these businesses of the contribution and importance of grid to future business. Take a look through the presentations, I am sure you'll find much of interest. As always, there is more about many of these cases on the GCN website.

Green IT

2007-05-01T08:05:00.000-07:00

I've been digging more and more into the issue of sustainability and the responsibility of the Data Centre for energy consumption. In the UK this is estimated at around 1.5% of gross domestic consumption, which seems a lot to me. However, when you consider the average utilisation of servers running around 15-20% (which is probably optimistic) its actually laden with a fair amount of wastage. Assuming that consumption is linear in nature - which it is probably not. When you consider that powering a box takes about twice as much power as cooling it the statistic gets worse. If you then consider that older boxes may be idle and yet running at 85% consumption, it gets worse still. At least newer processors use only around 25% in idle state and can now be switched off altogether and remotely powered up and operational in less than 15 minutes. This is a story which will run and run, because I know that the IT Directors and Data Centre managers out there are quite passionate about power. Its now consuming up to 40% of their budget and, we can see from the above, to little good effect. Its something we need to get to grips with. How?
Well, using virtualisation allows server power to be maximised in a fewer, more efficient boxes for end users; server virtualisation allows older, more expensive, boxes to be replaced by newer more efficient servers - and the Xen hypervisory looks good at the price for only single digit overhead against double digit performance improvement.
There's some smart ideas around from HP on power management in the data centre, combined with grid middleware and system management workloads can be distributed across the network and usage levels optimised to your choice. And while we're there, are you sure you need to own your own infrastructure? There's always the opportunity to leverage someone else's investment in carbon neutral infrastructure for hosting some of your applications or to provide extra capacity for business continuity.
Multi-core processors seems to prosper best in virtualised environments, at least that's one way of running application tasks in parallel, simply replicate them. Works well if transactions are self contained. A little more challenging if not. But there are ways of clustering which will work for more traditional data base configurations, see Oracle 10G. Course, it would be good if we could dust off those old parallel programming skills and redesign applications for these environments. See the BCS Parallel Processing Specialist Group for some folk who understand this question!!
Finally, its such a shame that we have so much AC:DC conversion lying around, Google, amongst others, have suggested a minimum standard of 80% efficiency for all such exercises. Does this cost you much? Well a recent experiment successfully ran a data centre on DC power only, just one conversion, at a saving of 20% of energy consumed! Run the maths, there's plenty of opportunity here to start reducing that power bill.

Setting course for the future ...

2007-04-03T05:57:00.000-07:00

The Grid Computing Now! team is embarking on the next stage of their project, looking forwards to the next wave of innovation to hit the ICT streets. There are some obvious trends to track, in no special order:- Virtualisation; Multi-core; System Middleware; Service oriented Architectures and Web Services. All this against a background of increasingly pervasive networking and growing storage capacity. That much is pretty obvious, but where are the disruptors which enable new opportunities - such as wireless communications and mobility; or personal computing. What are the new frontiers of application which will change the world we live in for the better? How will we manage in a less secure world; with higher and higher yields of crops required to feed us despite a changing climate? How will we travel, will we need to, will we have to pay to do so on public roads? How will we improve the situation of the vulnerable and elderly in society, especially those without computer skills in a world where many transactions will be made on-line? Is there a dividend to be obtained from pervasive computing - if so what and where?

If you've any bright ideas about the next wave of innovation in computing, particularly in the application of computing to our societal needs, I would be interested to hear your thoughts. We have an event at the end of May which is aimed at exploring this whole area more systematically. But, your ideas would be equally of interest if you could post them against this blog.

Looking forwards to hearing from you!

Gathering Momentum!

2007-03-23T03:24:00.000-07:00

It's been a while since I sat down to update my blog, part of that was a change in the Blogger system which I use, which resulted in the URL of the blog changing, the new version of IE apparently found this out and normal service has been resumed! But the main reason is that its been a busy time, a lot of water flowing under the bridge and some exciting things emerging!

Firstly, our webinar series has really got some momentum and the last event in particular shed light on the challenge of maintaining large scale databases in a distributed environment. A problem that the NHS has in spades. The work of the IBHIS project is well worth taking a look at, if you are puzzling over how you might pull together the data from several databases into one view. It also is a more resilient and secure way of managing large infrastructures, allowing individual databases to be maintained by their owners and by the same token ensuring that only data which is permitted to be made public is seen elsewhere. A possible solution to the concern that 250,000, yes a quarter of a million, NHS employees might have access to patient medical records when the system finally goes live!

This led me to thinking of a different spin on this. Why shouldn't we own such data ourselves? Either directly or via an agent. This would allow an individual to grant access to a relevant selection of data to someone authorised to use the data, rather than leaving the whole shooting match under someone else's control! This is not a new thought process as I discovered when I asked the question in the IBHIS Webinar, but the IBHIS data federation scheme could cope with this. Such a scheme would allow us to reveal our personal details to our doctor during a consultation. Grant access to specific data about us, appropriately obscured, for medical research; provide for emergency access to your medical data from an accredited centre; and prevent wider abuses of personal data being made fully accessible, perhaps by an agent of your insurance company while they are considering a proposal. I'm sure that there are down sides to this from the perspective of 3rd party access to medical records. But then, after all, whose data is it? See www.gridcomputingnow.org to find the webinar.

For those of us who are terminally doubtful about the adoption of such exciting new technologies, a meeting late in February was of some reassurance. The Financial Services community met in the City to consider the development of a generic description of grid middleware functions to enable the abstraction, or hiding, of propietary solutions from the customer implementation. This is intended to allow for multiple supplier's solutions to be implemented across the increasingly large deployments of "grid" clusters, now numbering in the several thousands. To their credit the leading suppliers were at the table gently teasing each other while playing along patiently with their customers. Watch this space over the next few months, we could be looking at some new inter-operability in this vital area of the Grid. If you are interested in participating do contact me: ian.osborne@intellectuk.org.

Finally, these and many other exciting developments will be discussed at the Open Grid Forum's 20th Conference event which will be held in Manchester, May 8th to 11th. During this event, Grid Computing Now! will host a Grids Mean Business Track on the 8th and 9th, which will feature many of our friends from the case studies which we have developed in past 2 years. Further details at http://www.ogf.org/OGF20/events_ogf20.php register soon, there is an early bird discount available for the Grids Mean Business Track which ends this month. Type "GMB" into the appropriate field on the registration form. More details at www.gridcomputingnow.org

Welcome 2007! Let's make a difference!!

2007-01-22T02:21:00.000-08:00

I always like to reflect on the past year and immerse myself in future looking publications around the New Year. This year was no different and I resolved to focus on making one or two things change as a result of the activities of Grid Computing Now! in 2007. The first issue to be faced is that of Software Licensing. This issue has hung over the industry for many years, the issue being that the irresistible force of software vendors wanting to exercise control of user behaviour has been resisted by the immovable mass of reality in the guise of customer business needs. Now, Open Source movement aside, no-one seems to be disputing the need to pay a fair price for a valuable piece of software for use in business.

The issue appears to be all about the mechanisms by which the supplier is paid. On the one hand we have software in the enterprise which is widespread on every desk, such as Office; Outlook; Explorer; etc.. On the other we have a smaller penetration of higher value tools for use in engineering design; manufacturing planning; mathematical modelling and simulation; etc.. With the advent of virtualisation at server and network level and the introduction of multi-core systems the problem has just been magnified substantially. Each of the vendors is trying different mechanisms to try and manage their way around the fact that their software may be used in a more widespread manner than licensed. This could become a tail-chasing proposition of epic proportions to keep up in these rapidly evolving virtual environments!

It seems that there ought to be easy ways of managing this problem and I intend to find out what you, the users and readers of this blog, think those easy ways should be. I'm intending to canvass 100 opinions in the lead up to the Grids Mean Business event at the Open Grid Forum conference which will be held in Manchester on May 8th and 9th, see http://www.ogf.org/OGF20/events_ogf20.php
How can you help? Well let me know via email the answers to the following questions:-
1) Is software licensing a problem for you?
2) Why or why not?
3) If you could wave a magic wand what would your ideal solution be?
4) Any other thoughts?
Send your message to info@gridcomputingnow.org and watch this space for updates on my progress to 100 opinions to make a difference in 2007!

The 451 Group Enterprise Computing Strategy Summit, New York, December 11th and 12th 2006

2006-12-13T04:46:00.000-08:00

Just back from a hectic trip to New York to join the 451 Group and a myriad of advanced customers and industry leaders debating the merits of Grid, Virtualisation and issues surrounding in several industries. Bottom line from the event is that Grid is real (virtualisation notwithstanding), its in widespread use in Financial Services; Engineering; Pharmaceutical and of great interest to the Telecoms industry looking to capitalise on pervasive IP networks to becaome service providers.

There were many highlights in the event which featured an eclectic mix of, analyst scene setting; advanced user case studies; lively panel discussion of issues du jour. For me, I found the ideas of market making for grid service provision emanating from the Financial Services industry to be quite compelling, I had something similar from a different source in that industry earlier in the year. The basic idea is to offer capacity "on demand" at a price which reflects market forces. On reflection though, this idea may provide the basis for a real utility offer.

On that subject Utility computing was discussed thoroughly over the two days. Am I wrong in thinking that perhaps the only real way of defining this is in the manner of the ability of a user to reserve capacity ahead of time and release unneeded capacity dynamically? This would be analagous to common metaphors in the power grid. There was a general consensus that outsourcing and the like (e.g. Software as a Service), was much more along the lines of business process outsourcing or functional integration of capability , i.e. bringing CRM or ERP capabilities into the organisation. This may become a very persuasive offer to organisations of all sizes in the future, assuming that they haven't outsourced the entire IT department and lost the ability to think through the IT contribution to their businesses.

Obstacles to this brave new world are the usual suspects: Security (although there is a strong line of argument that your biggest risks are currently within the firewall already!); Change Management, how would you introduce someone else's solution with noone in house to champion and train the staff?, and service level management!

On the grid level there was debate about the G-word, but none about the value. It seems that everyone is doing it and there was discussion about the contribution of open-source middleware for workflow management and hypervisor which tended to fly in the face of current industry assumptions that everyone will buy proprietary solutions. The prediction from the assembled experts was that open source will become de facto standards for key capabilities in managing an heterogeneous network of computers, and that this will be drive the standards process.

On the topic of Standards, Paul Strong, e-Bay, made a powerful plea for leading industrialists to engage with the standards bodies, Open Grid Forum, OASIS, etc., in order to bring some focus and priority to bear in these bodies. Paul is a vice-president of OGF and is keen to raise the debate from the industry side. Note, Grid Computing Now! is planning an Industry track for the OGF 20 meeting in Manchester in May (8th and 9th, 2007) for those interested in learning more!

From the 451 Group, William Fellows and Steve Wallage led the way with the Grid Adoption Research Service (GARS) reports, they introduced a nice 5 layer model explaining the phases of adoptive behavour that they have seen to date. Rachel Chalmers nearly stole the show with a compelling and energetic preview of her investigations into Virtualisation, major changes are taking place on the supply side with Intel and AMD incorporating key VM features into their designs looking forwards. VMWARE and XEN look well positioned for a healthy licensing income!

Jim Davis gave an overview of some grid activity in the media sector, of interest to all; and we held discussions on Licensing, yours truly made his pitch for gathering user requirements, watch this space, security and open source software.

GridWorld2006

2006-09-19T08:52:00.000-07:00

Tis the Conference Season, last week, Grid World 2006 in Washington DC. The first combined conference featuring the Open Grid Forum, a confluence of the Global Grid Forum and the Enterprise Grid Alliance, and Globus World, the original Grid movement led by Ian Foster and Carl Kesselman and those at Argonne National Labs in Chicago.

As usual my interest in the conference is to find commercial implementation and adoption experience and this conference contained some gems and some more of the same. Gems which I enjoyed included Paul Strong's expose of the underpinning infrastructures at eBay. A truly global scale enterprise delivering web services to nearly 200M active users worldwide. Several interesting facts emerged: eBay develops its own tools to manage the infrastucture in a grid style. They are able to build a new instance of the many eBay websites in a remarkably short number of days - allowing them to roll forwards at speed, and have an highly robust architecture aimed at ensuring that they don't suffer the revenue loss of $3K per minute which would accrue if service is interrupted.

I encountered one or two interesting snippits about eBaying behaviours around the world. In some Asian countries people take pictures of goods on the shelf in stores, post them and offer them for sale, presmably at a premium. Rushing off to purchase only when an order is received! Another practice is to offer a food delicacy online to the highest bidder. For those interested Paul is Distinguished Research Scientist at eBay Labs, previously at Sun Microsystems.

Presentations from the conference are available at the delightfully named http://gridworld.pistonbroke.com/. I can recommend Andy Mulholland's vision of a service enabled collaborative world. Of course, we presented our own GCN! Workshop on Knowledge Transfer with the willing assistance of Mark Parsons. The workshop was well attended, by the standards of the Community Programme and a lively discussion ensued on the matters of knowledge transfer. The bottom line from the discussion revolved around the usual problem: how do we make grid computing relevant and exciting for the business community. That's close to what keeps me awake at night!

Another gem was the product offerings of United Devices, I liked their approach to hetergeneous networks of computers and they have a sophisticated approach to assessing traffic and activity which enables the gathering of data for use in service level management. Critical for those service providers wishing to offer utility capability to their clients.

The attendance at the event was a little thin, the very large environs of the Washington Convention Center were not so friendly. But my abiding impression of the event was that the programme was stronger than the previous Autumn, but that it is very much the supply side of the industry which is getting to grips with the technicalities of grid infrastructure. I wrote some of these thoughts in an interview for GridToday ahead of the EGEE'06 conference in Geneva next week - but that's another story. For a more comprehensive review of the conference see GridToday's coverage at http://www.gridtoday.com/gridworld/06/index.html .

A special delight for the UK team present was the award of the GridToday Editor's Prize for the e-Science programme. Unfortunately they were rather busy in a meeting planning the next major international event on the OGF calendar, OGF20 will be held in Manchester during the week beginning May 7th 2007. GCN! will be presenting a business track at the event. Contact Dave Berry to share your interests or potential offer to help. Personal delights during my visit were the glimpse of the White House from the street, I'm a big fan of the West Wing, and the new cuisine of Rosa Mexicano at 7th and F streets. A restaurant where they make the guacamole fresh at the table.

WWW2006

2006-06-14T09:08:00.000-07:00

The Annual W3C conference WWW2006 took place in Edinburgh, beginning Monday, May 22nd. With NeSC based colleagues, Gillian Law and Dave Berry, the GCN! team was well represented at this conference and you will see several pieces of commentary and reporting emerging on the site. The major theme of the conference was Semantic Web and the event was put together by the team from Southampton University, Professors Nigel Shadbolt; Wendy Hall and GCN! Competition Judge, David DeRoure.
It was a large conference attended by 1200 folks from all around the world and featured the great and the good from the field of the Web and Computing including: Sir Tim Berners-Lee; Professor Tony Hey, another friend of GCN!, now Senior VP for Technical Computing at Microsoft; and many others. There were several interesting sessions featuring Grid, Security, Service Oriented Architecture (SOA), Healthcare and so on. My key takeaways from the conference lay mainly on the importance of semantics in navigating the future web; the nature of collaboration between knowledge workers and how important it will become to define context (ontologies, taxonomies and semantics) if we are to be able to work together; the pervasiveness of security issues in the internet world - especially the newly discovered commercial antics of hackers (your money or your site's dead!) and the huge potential for web services if we can crack the code and allow discovery; governance and payment schemes to work. Take a look at the site for more.

Future of the Data Centre 2006

2006-06-14T08:16:00.000-07:00

Completing an interesting sequence of recent events, I attended and spoke at the recent Infoconomy Future of the Data Centre (FoDC) conference in London on June 6th and 7th. I spoke on the emerging consensus for enterprise IT architecture, comprising Virtualised Infrastructure; Service Orientated Acrchitecture and Integrated Web Services. Leveraging thought processes accumulated from several sources over the past month or so. The most interesting and newer insights came from WWW2006 where the theme was most definitely, the Semantic Web, or Web 2.0, see my previous post, and a really interesting discussion with Andy Mulholland, CTO at Capgemini, who shared his thoughts regarding the evolution of computing infrastructures to support a web services based collaboration space between organisations. The talk is posted on the Grid Computing Now!, and at the Information Age FoDC websites. Comments very welcome.
Apart from my inputs at the beginning of the conference, I was delighted to hear further evidence for the key steps in developing the enterprise IT infrastructure of the future from colleagues who spoke of Virtualisation; Service Orientation and Utility computing during the first day.
The second day was primarily about practical IT issues, configurations for ease of management of knowledge workers; a lot about power provisioning and cooling issues to do with modern computing equipment in the data centre. By the way have you come across the Green Grid Alliance? This is an association of several IT industry suppliers who have taken up the challenge of reducing the power consumption of IT equipment.

AsiaGrid2006

2006-05-19T18:22:00.000-07:00

It's been my pleasure this week to attend GridAsia2006. Hosted by the National Grid Office in Singapore with plenty of support from the leading Grid vendors: Oracle and Platform in particular; the event has combined a meeting of CCGrid - the 2006 IEEE forum on cluster computing, GECON - a symposium focused on developing a market model for grid, ROCKS and a special meeting aimed at UK-Singapore collaboration in Science, focused on Grid Technologies. This last meeting was the icing on the cake of an interesting week for me. Yesterday, we were able to showcase the work of several of the UK eScience collaborative projects ranging from DAME/BROADEN from Rolls-Royce; White Rose Grid from York University; GOLD an ambitious plan to develop a virtual chemical engineering industry in the NorthEast of the UK centred around the work at Newcastle University and finally, the work in Cardiff University of Omer Rana, Deputy Director of the Welsh eScience Centre. Presentations from all are available at the GridAsia2006 website. http://www.ngp.org.sg/gridasia/2006/index.html

The Singapore Government is championing the use of Grid for three strategic industries:- Bio Technology; Digital Media and Aerospace, and the forces in the industry are marshalling their efforts around these themes led from the front by colleagues in the National Grid Office and Simon See of Sun Microsystems who leads an industrial research activity in Singapore based at NTU. The interest of the UK government is to broaden collaboration and leverage the experience and knowledge of the eScience programme in Singapore. Following the seminar, there were several lively discussions around possible areas of collaboration. Watch this space!

Grid Computing Now and Forever!

2006-05-04T07:53:00.000-07:00

Hi I'm Ian Osborne and this is my personal log of experiences which I have while responsible for the implementation of a UK project to promote the use of grid computing technologies. The project, Grid Computing Now! (GCN!) is a government funded Knowledge Transfer Network (KTN) with the goals of:-

creating awareness in the target sectors;
identifying obstacles and barriers to implementation;
preparing UK public and private sectors for adopting grid computing technologies;
leveraging the knowledge and experience gained in the UK eScience Programme.

The project started in February, 2005; was officially launched in July 2005 and our website www.gridcomputingnow.org was introduced in August 2005. We have a project team of 2 people based at Intellect, the UK Hi-Tech Trade Association, in Russell Square, London; and 2 more people based in Edinburgh at the National e-Science Centre, Edinburgh University. We also have a willing bunch of enthusiastic supporters from the UK Computing Industry; eScience community; commentators, investors and some genuinely experienced users in our project Advisory Council. A sister project is run by CNR Ltd, based in Bristol, and they provide support valuable support for the GCN! web-site and are focused on smaller businesses and the organisations which serve them around the nation.

As the project has developed we've come across a whole range of people with real experiences of Grid Computing in use, both from the academic community where it is big, and from the user community where it is growing fast in some sectors, established in others and unknown in some key places.

Our definition of Grid Computing Technologies is very broad, seeking to focus on the current industry themes of Virtualisation; Distributed job management; Service oriented architectures fo example. We run technical events to educate and inform, with a particular interest in "how to" rather than simply "what"! Our web-site provides an edited source of up to date news and a growing portfolio of user case studies written by our own in-house journalist.

You might be asking why? Well, it seems to us that we already have a lot of computing capability lying around, some worked hard - if not overstretched, but much of this under used and, frankly, wasted potential. This in spite of major challenges from parent organisations to IT to deliver more value! We believe that grid computing offers some clues as to how to both increase the value of existing investments and improve efficiency, and down stream to offer the opportunity to significantly add value through tying IT infrastructure to the organisations activities themselves.

How? Well check out our web-site for ideas, keep looking for posts to this blog and we'll see!