Background
My name is Jim Calise and I have the pleasure to be a member of the Bare Metal Cloud Product Management team. One of my main responsibilities is working with customers both internal and external to ensure their success using our infrastructure services. This role takes me in many different directions, but a common theme is demonstrating the developer experience, and how their current tools and practices can be used with Oracle Bare Metal Cloud Services to achieve their automation goals. Discussing and demoing our developer experience is without a doubt one my favorite parts of my job as I like to believe I am just a developer at heart who happens to enjoy speaking and evangelizing on our amazing services. My developer focus leads me down the path of hacking, and tinkering in my spare time to create demonstrations showcasing the power of our services API, and SDKs, and thus is the impetus for this blog. I will surely be blogging on a number of different topics over time, but today I am starting the first in a series of posts that will focus on how a developer can interact with our service through our API, and showcase how different developer tools, and CI/CD pipeline can be integrated.
Goal
I will walk you through the steps to get started with using our API, showcase an example of provisioning and configure with cloud-init a new compute instance , and promote some good practices in the process.
Let's Begin
To start I want to acknowledge the BMC Services Documentation and SDK teams. Two amazing groups of people I get to work with who have made my life incredibly easy when working with customers on developer automation.
A key design principle within our BMC Services is that everything you experience in the web console is accessible through our RESTful API. The web console itself is built upon the same public APIs you have access to. Most demonstrations you will run will start with the web console, as the web console is more visually appealing, and easier to follow as compared to running a command line script. Delivering this point will keep a broad audience of developers, operations, management, etc.. fully engaged.
Another key design principle within our Bare Metal Cloud Services is security. We are building a cloud infrastructure service with a focus on enterprise customers and this has led us to deliver a secure RESTful API that requires all requests be signed. The API requires the following
- Form the HTTPS request (SSL protocol TLS 1.2 is required).
- Create the signing string, which is based on parts of the request.
- Create the signature from the signing string, using your private key and the RSA-SHA256 algorithm.
- Add the resulting signature and other required information to the
Authorization header in the request.
This means that using curl, SOAP UI, Chrome Advanced Rest Client is not the best approach to interact with BMC service in a programmatic fashion. You can see you do not simply plug the URI into your tool of choice and provide an auth token in the header. If you want to use the RESTful API directly the steps are well documented here, where you will see you need to code the steps to generate the properly signed request.
You customer's view on using the API is rooted in the goal of wanting to automate service behaviors. Your customer's automation is not simply making a single API call, but rather building a parameterized script that creates a VCN, with an internet gateway, custom security list, and subnets, and then starting 1...* instances within the newly defined VCN's subnets. So, your customer wants to script/code their interactions, and providing a RESTful interface only is woefully inadequate. So, rather than requiring our customers to build the low level integrations to our RESTful API we provide a set of SDKs (Java,Ruby,Python) and a CLI to simplify the interaction with our services. I italicize requiring above because we understand that every customer's situation is unique and their are cases where coding directly against the RESTful API is a better option than using the SDK, so it is always your choice in how you want to interact with the API.
Pre-requisites
Do you have access to BMC tenancy? If not you can visit shop.oracle.com to get access, or contact your Oracle sales team.
Once you have access to a tenancy I suggest you familiarize yourself with our documentation available here.
The initial steps to get started are well documented, and to begin you need to setup your local laptop or development machine from which you will be running the SDK examples.
- Start here to follow the steps to collect the needed information from your tenancy, and generate the proper ssh key pair for your user who will invoking the API.
** The outcome of this step you should have your API user OCID, your public key uploaded, and the generated fingerprint ID, and your tenancy OCID.
- Create your local configuration file and populate it with the details you collected in step 1. Follow the documentation here.
** The outcome of this step is that your development machine is configured with your configuration file and you are ready to start using the SDK.
Example Config -
[DEFAULT]
verify_certs=false
log_requests=false
logger=
tenancy=ocid1.tenancy.oc1..aaaaaaaay7s6icq755xqlytpl33i7ysjzzb2kv3vk3itg5ilsxanrzqmsaha
region=us-phoenix-1
[ADMIN_USER]
# Subsequent profiles inherit values from DEFAULT.
user=ocid1.user.oc1..aaaaaaaa65vwl7zut55hiavppn4nbfwyccuecuch5tewwm32rgqvm6i34unq
fingerprint=72:00:22:7f:d3:8b:47:a4:58:05:b8:95:84:31:dd:0e
key_file=keys/admin_key.pem
pass_phrase=mysecretphrase
[Second_Tenancy]
user=ocid1.user.oc1..xxccccndychm6kxb34fledy6iectsupr44xdftqxwy4cxoqwj3nbsvjd2q
user=ocid1.user.oc1..aaaaaaaagnntws7i4dqaibnykpsx7ufqxq3bk4b7rkboelucehqz47smthoa
fingerprint=a0:36:45:21:af:bd:50:e6:22:85:58:fc:b6:5a:64:26
key_file=~/.ssh/id_rsa
fingerprint=a0:36:45:21:af:bd:50:e6:22:85:58:fc:b6:5a:64:26
tenancy=ocid1.tenancy.oc1..trmegsfefy7vovgqfr3glxkjttmqi3hz5sum3zexs6gz4kwfjsd5a
Install SDK
Next we need to install an SDK to interact with our service. You can download either Java, Ruby, Python or CLI here. Note in the documentation the specific API services that the SDK of your choice supports. As of writing there is a bit of variation across the SDKs in terms of support for Object Storage, and Load Balancer service, but over time these differences will go away. My examples going forward will be based on the Ruby SDK. Each of the SDKs provide documentation for setup and examples of usage to get you started. Please follow this documentation to get the Ruby SDK installed, and validate your install.
Understanding Instance Metadata
Each of the SDKs provide an example on provisioning an instance, and you will find that you provide your SSH public key to the instance via metadata. The compute service provides a meta-data service to each instance to provide data dynamically to an instance. You can read here how to access this metadata service from within your instance, and use curl to retrieve the data available to you by default, and anything custom you decide to pass in.
curl http://169.254.169.254/opc/v1/instance/
If you are familiar with AWS or OPC you have similar facility to customize your instances during boot. Custom metadata today can be provided as part of the create instance payload to provide the capability to dynamically customize your instance during the boot process. The images provided by BMCS you launch by default contain the Cloud-Init runtime that executes on boot of your server, and reads the from the metadata service looking for properly formatted cloud-init YAML document. Following the Cloud-Init documentation you specify a properly formatted YAML document that can do things like run shell scripts, execute commands, create users, execute Chef, etc...
Putting the pieces together - Provisioning and Configuring an Instance with Ruby SDK and Cloud-Init
We have a published workshop that you may be familiar with that walks you through the console experience of building a VCN, provisioning a server, and configuring that server manually from the command line to install Mongo, and the MeanJS stack. In the next few steps you will in a single API call provision a server via the Ruby SDK that will automatically configure the same application via Cloud-Init.
Step 1 - Define a directory on your local filesystem to contain the artifacts we are going to create.
Step 1 - Define your Cloud Init YAML. Create a file meanjs.yml within this directory and copy the following contents within your file.
#cloud-config
# vim: syntax=yaml
#
write_files:
- encoding: b64
content: W21vbmdvZGItb3JnLTMuMl0NCm5hbWU9TW9uZ29EQiBSZXBvc2l0b3J5DQpiYXNldXJsPWh0dHBzOi8vcmVwby5tb25nb2RiLm9yZy95dW0vcmVkaGF0LyRyZWxlYXNldmVyL21vbmdvZGItb3JnLzMuMi94ODZfNjQvDQpncGdjaGVjaz0wDQplbmFibGVkPTE=
owner: root:root
path: /etc/yum.repos.d/mongodb-org-3.2.repo
permissions: '0664'
packages:
- mongodb-org
- mongodb-org-server
- gcc
- gcc-c++
- wget
- git
runcmd:
- [ sh, -xc, "systemctl stop firewalld && systemctl disable firewalld" ]
- [ sh, -xc, "service mongod start" ]
- [ sh, -xc, "curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -" ]
- [ sh, -xc, "yum install -y nodejs" ]
- [ sh, -xc, "npm install -g bower" ]
- [ sh, -xc, "npm install -g gulp" ]
- [ sh, -xc, "git clone https://github.com/meanjs/mean.git meanjs" ]
- [ sh, -xc, "cd ./meanjs && npm install" ]
- [ sh, -xc, "cd ./meanjs && bower --allow-root --config.interactive=false install" ]
- [ sh, -xc, "cd ./meanjs && gulp" ]
2. Base64 encode the entire contents of the file. The Cloud-Init runtime expects Base64 encoded formatted content to be read from the metadata service. If you do not provide this format, Cloud-Init will not be triggered. We can do this in one of two ways:
a. Programatically from within Ruby or your SDK choice, where you read the file and encode it inline. I follow this approach below.
b. Use an online utility like https://www.base64encode.org/ and hardcode the results in your request payload.
3. Use Ruby SDK and create a CreateInstance.rb file within your directory.
require 'oraclebmc'
require 'base64'
require 'yaml'
my_config = OracleBMC::ConfigFileLoader.load_config(profile_name:ARGV[0])
computeapi = OracleBMC::Core::ComputeClient.new(config:my_config)
#Load Config
config = YAML::load_file(./meanjs.yml)
request = OracleBMC::Core::Models::LaunchInstanceDetails.new
request.availability_domain = ad
request.compartment_id = server_compartment
request.display_name = MeanJS_Server
request.image_id = imageID
request.subnet_id = subnetId
request.shape = server['server']['shape']
request.metadata = { ssh_authorized_keys: server['server']['ssh-key'],
user_data: Base64.strict_encode64(String(config))
}
begin
response = computeapi.launch_instance(request)
computeapi.get_instance(response.data.id).wait_until(:lifecycle_state, OracleBMC::Core::Models::Instance::LIFECYCLE_STATE_RUNNING,max_interval_seconds: 5,max_wait_seconds: 300)
vnics = computeapi.list_vnic_attachments(compartmentId,{instance_id: response.data.id})
network = vcnapi.get_vnic(vnics.data[0].vnic_id)
puts "Success!! Server Is Available - Public Ip: " + network.data.public_ip + " and Private Ip: " + network.data.private_ip
puts "SSH to the machine, and tail -f /var/log/cloud-init.log for server configuration details...MeanJS will be up shortly at http://" + network.data.public_ip + ":3000"
rescue OracleBMC::Errors::ServiceError => e
# wait_until might throw timeout or other errors.
puts "Error: " + e.to_s
end
end
Wrap Up
With your instance successfully provisioned you are on your way to using the Bare Metal Cloud Services SDK, and Cloud Init runtime to provision server. Our next blog will expand this example and walk through setting up the forthcoming Terraform provider for BMC, and provision this same server and configuration through Terraform.
The code and configuration files for this blog and all future entries are available for download from here.