Data Center World, Spring 2008

The following are notes that I made during two keynotes at Data Center World.  I didn't do a great deal of editing so please let me know if you have any questions!

Data Center World, Las Vegas

Keynotes

04-01-2008

 

Michael Manos,
Sr. Director of Data Center Operations, Microsoft
 

Microsoft Challenges

• Microsoft increases the number of servers 15,000 – 20,000 per month
• They double the number of servers each year
• Power
• Tens of megawatts of power usage

Why Power matters:

• Federal government will not be able to build power stations fast enough
• Sustainability regulations are coming
• How efficient are you and can you back up the numbers
• Monitoring will need to be in place
• Building a program to monitor and measure facilities is not an overnight process
• Cost of 1U server is much less than the power to run them.

Not solving the problems together?

Economics are changing

• Power bills may not be seen by DC managers
• Power is a more significant component
• 30% of DC managers don’t think power is an issue

Industry Challenges/Minefields

• Sustainability reporting/ efficiency reporting
• Data center inventory “Globally”
• Increasing power densities at the rack level
• Power densities per rack keeps growing
• 8 – 36kw per rack
• Power is driving the square footage costs
• Power Costs
• Green-washing
• Expertise shortage
• Organizational structures
• Increasing capital cost barriers
• Innovation hoarding
• Heterogeneity versus homogeneous mindsets

 

To drive a .1 PUE decrease, decreases 7 million kw (Microsoft)

• MS could spend up to $500M per data center

$11 – $18M cost for construction of a new data center and will grow 10% per year.

• Not based on housing industry
• Completely different skills involved
• Mostly electrical and mechanical (80%)

Data centers at MS

• Bring Moore’s law to the data center
• Unfortunately the data centers have not kept up
• Challenged Microsoft team to bring it to the DC
• Innovation hoarding
• MS will share its knowledge

 

Data Center Packman  (Typical DC Costs)

• 2% Land
• 9% core and shell
• 7% architecture
• 82% Mechanical and Electrical

Density Questions

• If average rack now is 16kw how can we plan for the future
• How do we balance for high density VS less densities
• OK, if you are consuming all the power

Microsoft Best Practices

• Building 500,000 sq ft data centers on average
• 40k – 100k systems per building
• Highly optimized
• 10k – 20k growth per month
• 35 people spread across MS to manage deployment
• Gone is the days of unpacking servers
• DRIVING AUTOMATION IS KEY
• Discovery of the servers and imaging
• Site selection is critical for new data centers
• MS is growing their numbers
• Some are consolidating
• Carbon footprint is already set if you currently have a data center
• Look at where the internet is and power pricing
• Environmental
• How can it be optimized in certain areas
• Going green is good for the bottom line
• Construction costs
• IT costs and availability in area
• Power cost in the area
• Where is the carbon emission factor in the area
• Mobile users
• Tax climate or breaks

 

PUE VS DCIE

• MS uses PUE and are reciprocals of each other
• Engineers love DCIE
• PUE shows the inherent overhead
• Doesn’t matter but must drive efficiency
• MS drives these targets internally

If I can half the distance by 1 each year brings Moore’s law into play

By 2012 PUE below 1.2 for Microsft

MS uses a tool they call scribe to evaluate their efficiency

• On average of 1M points of monitoring per each data center
• CPU, memory, overhead temps, etc…
• Sensors can tell what is going on in each rack and replay events in the DC
• Measure an environment as
• Cleaning the roof on one of MS’s DC’s was able to reduce PUE (very large facility)

Top ten best practice that have been published by MS

MS Datacenter in Chicago is modular in design

• They will have standard containers that will hold up to 2,000 server that they back into a data center and hook up power, liquid for cooling, network and ground.  Everything else is pre configured at the vendor site

Could we use IE students or project to model a data center build??

What’s next

• MS participating in Energy Star Forum
• Share with industry

 

Hu Yoshida

VP and CTO – Hitachi Data Systems

 

Sustainable Data Storage is more than Green Technology

Storage can’t be moved since it’s stateful and will be destroyed if not handled properly.  Typically server are stateless since they can be plugged and unplugged without loss.

The Restriction of Hazardous Substances (RoHS) Directive 

Move data off current storage

• Consolidation
• Fewer footprints, fewer LOCATIONS
• Increase utilization
• Of server, storage, network, power and cooling resources
• Eliminate redundancies
• Delete, compress, dedupe, store and process once
• Reduce the working set
• Archive stale data from production systems
• Technologies coming that will scale to petabytes

 

Management software

• Common user interface
• Efficient management
• Better resource utilization
• Reduce staff training
• Improved TCO

Archiving as an environmental tool

• 70% of data over 60 days old is rarely referenced again
• This is toxic waste, wasting time, money and fossil fuels
• Keep one gold copy in a single place

Optimizing storage capacity

• Thin provisioning
• Unallocated space is huge
• Growing storage as needed, giving people what they want but not allocating it until it is touched or provisioning only when used

Thin provisioning coupled with virtualized disks you can gain up to 60% efficiency

Virtualization can help move storage as needed without disrupting anything

Tiered storage reduces power and costs

Not all data needs same performance

Intermediate copies used for backup

Massive array of idle disks (MAID)

De-duplication

 

Which storage is more efficient

• Modular storage
• Monolithic storage
• Since there are fan’s at the top and bottom instead of in each module this storage tends to be more efficient

Look to Storage Neworking World (SNIA) http://www.snia.org for metrics on storage efficiencies

ISO 14001 references sustainability