Cassandra in Azure

CASSANDRA AND AZURE RESOURCE MANAGERHow it works and how to get it moving

CARLOS ROLOFEBRUARY 2016

WHO AM I?• Cassandra Consultant for Pythian• I’m all about Distributed Systems• Certified Datastax Architect• Cassandra MVP• Programming since 1997• Cassandra DBA since 2001• Previous experience:

– INESC/ID– DRI– Portugal Telecom– Leaseweb CDN

ABOUT PYTHIAN• 18 Years of Data infrastructure

management consulting

• 200+ Top brands

• 6000+ databases undermanagement

• Over 400 DBA’s, in 29 countries

• Top 5% of DBA work force,– 9 Oracle ACE’s, – 2 Microsoft MVP’s,– 1 Cassandra MVP

• Oracle, Microsoft, MySQL,Datastax partners, Netezza,Hadoop and MongoDB plusUNIX Sysadmin and Oracle apps

© 2014 Pythian Confidential4

AGENDA• Azure• Azure and Cassandra• Azure Templates• Automation• Takeaways• Q&A


AZURE• Microsoft Cloud Service• 53 different services• 27 Locations (22 Active)• CLI offerings for Linux, Mac, Windows


AZURE RESOURCE MANAGER• Azure Resource Manager (ARM) is a

“package”– You assign resources to your Resource Group– You can track all the resources from a single

point, including:• Billing• Monitoring• Access Control• Template based provisioning


AZURE INTEGRATIONS• Windows: Powershell, Azure-Cli, Visual

Studio• Linux, Mac: Azure-Cli• + Web Interface• + Rest API

• Level of integration– Powershell & Visual Studio >>> Azure-Cli


AZURE AND ME• It has been an uphill fight!

– But I’m getting more and more out of it, and I like it!

• But it was only one way to do things…– Then “classic” and “arm”

• Powershell was good…– Stuff broke between 0.8.x and 1.0.x

• Let’s use cli then!


AZURE AND ME(2)• Documentation,

Features…


AZURE AND ME(3)• Fully migrated from AWS• 3 Cassandra Clusters in it• Automated* deployments

– ARM and Classic are just a one-liner way.• And it defaults to Classic!

– Deleting resource groups sometimes gets weird• Some UUID is not allowing you to do whatever• Can’t find the UUID anywhere


AZURE AND CASSANDRA• Why?

1. Instance sizing: D4, D5, G2 and G32. Marketplace: Few clicks N node cluster (DSE)!3. Integrations4. Lots of Locations5. Latency, storage latency!6. Network!


AZURE AND CASSANDRA (2)• And the downsides?

– Storage has a lot of stuff going on,– Throughput… (YMMV)– The web portal takes time to update


AZURE AND CASSANDRA (3)• 8 clicks from 0 to “how-many-nodes-you-

need”!• Took me 4 or 5 min to go through all that!


AZURE TEMPLATES• This is where automation gets in!

– Painfull to learn, but…– … lots of examples available (github)– Visual Studio makes wonders here!


AZURE TEMPLATES (2)Seriously, if on windows, use Visual Studio!


AZURE TEMPLATES, THE GOOD• Easy to use• C# + Visual Studio + Templates• Github + azure.deploy.com + Templates• Lots of examples available!• https://feedback.azure.com


AZURE TEMPLATES, THE BAD• Did I told you how painful it was?• And look at all the documentation I can’t

find!• All those functions…


AZURE TEMPLATES, AND THE UGLY• Testing?

– The only way to test is to deploy…


AUTOMATION• Linux, Mac

– Azure-cli• Windows

– Powershell– Azure-cli

• Azure Automation Service


AUTOMATION (2)How do we make this get Cassandra automated?• “commandToExecute”• You can have your scripts in Azure

Storage, github or make them copied over to your VM

• It can be set to variables and things can get complex fast.– But it is the best way to do it!


AUTOMATION (3)

Provision

• Configure VM’s• Deploy VM’s and remaining Infrastructure

Install

• Install OS Packages• Install Cassandra

Configure

• Set configuration• Start the service


AUTOMATION GOTCHAS• SSH-Keys…

– Read the docs, in the past some weird problems appeared.

• Lots of resources,– For powershell!

• Getting handle on those “commandsToExecute”

• Read the logs!!


TAKEAWAYS• You’ll need patience if you’re coming from

AWS– You will go through a lot of trial and error.– Documentation and examples are scarce

• Azure is moving really fast– Can’t remember the last day I didn’t have an

update!– And can break stuff.


TAKEAWAYS (2)• Beware of storage!• Setting up domains• Lots of tools if you are on Windows.• You can deploy not knowing anything

about Cassandra!

Q & A!• Twitter: @cjrolo• LinkedIn: linkedin.com/carlosjuzarterolo• Blog: blog.pythian.com/carlosrolo

Cassandra in Azure

Technology

Transcript of Cassandra in Azure