vRealize Operations - Using Custom Properties with Python

Recently, a customer wanted to document some data that isn't natively collected for their hosts.  With vRealize Operations, there is a custom properties process discussed in the documentation.  Instead of adding custom properties through that process, I thought I would go through the API process in this blog.

For this exercise, I'm using Python with a Flask framework.  Flask makes HTML and Python much more practical.  My introduction to Flask was this project.  The structure I used for the app is very basic.

I deployed a virtual machine via vRealize Automation 8 to build and test the app.  You don't need to automate the deployment of an environment.  You simply need somewhere to use python, install Flask and its accessories and access a vROps server.  The Flask development environment will provide you with a web service to test your application.

My base Linux install is from a network boot ISO with a minimum install.  I tested with CentOS 8, Debian 10 and Ubuntu 19.  I then added the packages needed.  In my environment, I'm using Cloud-Init during the automated deployment.  The 'packages' and 'runcmd' section below will give you a list of software to install on a Linux resource.  I disabled the firewall during the install.  You may need the firewall in your environment.  Use your best judgement.

As I was working on this project, I thought about the ability to showcase more than just custom properties.  You'll be able to see how to acquire a vROPs token and then use the token through the process.  You will see how to query for all the adapters with all of their associated resources.

Keep in mind that this is the first time I've used Flask.  There are always other methods to accomplish the same tasks.  If you use your browser's back button, you may have to update the selection lists.  The intent is to show how to use APIs to update and create Custom Properties on resources.  Be aware that property names cannot be removed once they are added to the resources.

If you're in need of another method using vRealize Orchestrator, this blog is impressive: https://www.stevenbright.com/2019/06/add-custom-properties-to-vrealize-operations-using-the-rest-api-and-vrealize-orchestrator/

You can download the code seen below.  Extract with tar -xzf <filename of the download>

Start the app by switching into the Flask directory (the structure is discussed below) and execute: 'python app.py'

Once the app is running, use your browser to access your IP address on port 5000 (i.e. http://your-ip-address:5000).  You'll see the following in your browser:

Authentication

The authentication is written only for local accounts.

Adapter and Resource Kind Selections

When you select an adapter, the resource kinds associated will update accordingly.

Resource Selection

Select a resource to add the property and value.

A portion of the selected resources properties.

The ability to add a custom property name and value

Enter the property and its value.  This is a string entry.  Once you submit the data, it takes about 15-30 seconds before a subsequent query shows the information in the property table.  In the python code, I'm looking for none empty strings in the property name and property value before submitting to vROps.

I've include a link to the files in a compressed .gz file.

To Get Started:

You'll need a building/test environment with he following packages (I used Linux):

    - python3
    - python3-pip

Once the packages are installed, execute the following commands:

    - sudo systemctl stop firewalld
    - sudo systemctl disable firewalld
    - sudo pip3 install flask
    - sudo pip3 install jinja2
    - sudo pip3 install requests
    - sudo ln -s /usr/bin/python3 /usr/bin/python

If you are using automation in your lab, you can automate the building process with the following blueprint taken from my vRealize Automation 8 environment (change the image to match your image names):

name: Flask-Jinja2
version: 1
formatVersion: 1
inputs:
  VmName:
    type: string
    description: Will also be the hostname of the FQDN.
    title: VM Name
  ImageSelect:
    type: string
    description: The operating system version to use.
    title: Operating System
    oneOf:
      - title: CentOS Linux 7
        const: CentOS7
      - title: CentOS Linux 8 (Core)
        const: CentOS8
      - title: Debian Linux 10 (Buster)
        const: debian10
      - title: Ubuntu 18.04.4 LTS
        const: Ubuntu18
      - title: Ubuntu 19.10
        const: Ubuntu19
  CPU:
    type: integer
    enum:
      - 4
      - 2
      - 1
    description: Requested vCPU Count
    default: 1
    title: Requested vCPU Count
    minimum: 1
    maximum: 4
  Mem:
    type: integer
    description: Requested Mem (GB)
    title: Requested Mem (GB)
    minimum: 1
    maximum: 8
    default: 2
    enum:
      - 8
      - 7
      - 6
      - 5
      - 4
      - 3
      - 2
      - 1
  username:
    type: string
    title: Username
    default: testUser
    description: Create a user during deployment
  password:
    type: string
    title: Password
    default: VMware1\!
    encrypted: true
    description: Password for the given username
resources:
  Cloud_vSphere_Machine_1:
    type: Cloud.vSphere.Machine
    properties:
      image: '${input.ImageSelect}'
      newName: '${input.VmName}'
      cpuCount: '${input.CPU}'
      totalMemoryMB: '${input.Mem*1024}'
      networks: []
      cloudConfig: |
        #cloudconfig
        hostname: ${input.VmName}
        fqdn: ${input.VmName}.thewhiteshouse.net
        repo_update: true
        repo_upgrade: all
        package_update: true
        package_upgrade: true
        ssh_pwauth: yes
        disable_root: false
        chpasswd:
          list: |
            ${input.username}:${input.password}
          expire: false
        users:
          - default
          - name: ${input.username}
            lock_passwd: false
            sudo: ['ALL=(ALL) NOPASSWD:ALL']
            groups: [wheel, sudo, admin]
            shell: '/bin/bash'
        packages:
         - python3
         - python3-pip
        runcmd:
         - echo "Defaults:${input.username}  !requiretty" >;>; /etc/sudoers.d/${input.username}
         - echo root:VMware1\!|sudo chpasswd
         - history -c
         - systemctl stop firewalld
         - systemctl disable firewalld
         - sudo pip3 install flask
         - sudo pip3 install jinja2
         - sudo pip3 install requests
         - sudo ln -s /usr/bin/python3 /usr/bin/python

Once you have a development environment, create a directory for your application. For me, I created a directory named "Flask" with some sub-directories named "static" and "templates" as follows:

Flask

├── app.py
├── static
│   ├── favicon.ico
│   └── styles.css   
└── templates
    ├── adapter.html
    ├── index.html
    ├── properties.html
    └── resources.html

You don't need the file 'favicon.ico'.  It's the icon you see when you bookmark a page or the image seen in your browser's history or tabs.  The files needed for the application:

File: app.py

from jinja2 import Template
from flask import Flask, render_template, request, session, redirect, url_for
from uuid import uuid4
import cgi, cgitb, requests, json, time
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
cgitb.enable()
app = Flask(__name__)

@app.route("/",methods=['GET', 'POST'])
def main():

  message=''

  if request.method == 'POST':
    srvName = request.form.get('srvName')
    usrName = request.form.get('usrName')
    usrPass = request.form.get('usrPass')

    baseURL = "https://" + str(srvName)    
    tokenURL = baseURL + "/suite-api/api/auth/token/acquire"
    authJSON = {"username": usrName,"authSource": "Local","password": usrPass,"others": [],"otherAttributes": {}}
    urlHeaders = {"Content-Type":"application/json","Accept":"application/json"}
    response = requests.post(tokenURL,data=json.dumps(authJSON),headers=urlHeaders,verify=False)
    if (response.status_code == 200):
      session['token'] = "vRealizeOpsToken " + response.json()['token']
      session['url'] = baseURL
      session['user'] = usrName
      session['pass'] = usrPass
      return redirect(url_for('adapter'))
    else:
      message = 'invalid credentials'

  return render_template('index.html', message=message)

@app.route("/adapter",methods=['GET', 'POST'])
def adapter():
  if request.method == 'GET':
    baseURL = session.get('url',None)
    adapterURL = baseURL + "/suite-api/api/adapterkinds"
    authToken = session.get('token',None)
    dataJSON = []
    urlHeaders = {"Content-Type":"application/json","Authorization":authToken,"Accept":"application/json"}
    response = requests.get(adapterURL,headers=urlHeaders,verify=False)
    if (response.status_code == 200):
      dataJSON = response.json()
    else:
      print(response.status_code)
  elif request.method == 'POST':
    session['adapterURL'] = str(request.form.get('adapterURL'))
    session['adapterName'] = str(request.form.get('adapterName'))
    session['resourceKindName'] = str(request.form.get('resourceKindName'))
    return redirect(url_for('resources'))
    
  return render_template('adapter.html', data=dataJSON)

@app.route("/resources",methods=['GET', 'POST'])
def resources():
  if request.method == 'GET':
    baseURL = session.get('url',None)
    authToken = session.get('token',None)
    adpKind = session.get('adapterURL',None)
    resKind = session.get('resourceKindName',None)
    resKindURL = baseURL +  adpKind + "/" + resKind
    resourcesURL = resKindURL + "/resources"
    dataJSON = []
    urlHeaders = {"Content-Type":"application/json","Authorization":authToken,"Accept":"application/json"}
    response = requests.get(resourcesURL,headers=urlHeaders,verify=False)
    if (response.status_code == 200):
      dataJSON = response.json()
    else:
      print(response.status_code)
  elif request.method == 'POST':
    session['resourceID'] = str(request.form.get('resourceID'))
    return redirect(url_for('properties'))
    
  return render_template('resources.html', data=dataJSON)

@app.route("/properties", methods=['GET', 'POST'])
def properties():
  message=''
  baseURL = session.get('url',None)
  authToken = session.get('token',None)
  resourceID = session.get('resourceID',None)
  adapterID = session.get('adapterName',None)
  urlHeaders = {"Content-Type":"application/json","Authorization":authToken,"Accept":"application/json"}
  if request.method == 'GET':
    resourceURL = baseURL +  resourceID + "/properties"
    dataJSON = []
    response = requests.get(resourceURL,headers=urlHeaders,verify=False)
    if (response.status_code == 200):
      dataJSON = response.json()
    else:
      print(response.status_code)
  elif request.method == 'POST':
    customName = str(request.form.get('customName'))
    customValue = str(request.form.get('customValue'))
    if customName.strip() == "None" or customValue.strip() == "None" :
      return redirect(url_for('properties'))
    if not customName.strip() :
      return redirect(url_for('properties'))
    if not customValue.strip :
      return redirect(url_for('properties'))
    custData =  {
                  "property-content":[
                    {
                    "statKey":"CustomProperty|" + customName,
                    "timestamps": [ int(time.time()*1000) ],
                    "values":[ customValue ],
                    "others":[],
                    "otherAttributes":{}
                    }
                  ]
                }
    postURL = baseURL + resourceID + "/properties"
    response = requests.post(postURL,headers=urlHeaders,data=json.dumps(custData),verify=False)
    return redirect(url_for('properties'))
   
  return render_template('properties.html', data=dataJSON, message=message)
  
if __name__ == "__main__":
  app.secret_key = str(uuid4())
  app.run('0.0.0.0')


File: index.html

<html dir="ltr" lang="en">
  <head>
    <meta charset="utf-8">
    <link href="static/styles.css" rel="stylesheet" type="text/css"></link>
    <link href="static/favicon.ico" rel="shortcut icon"></link>
    <title>Authenticate</title>
  </head>
  <body>
    <div>
    <form action="" method="post">
      <label for="srvName">vROps Server</label>
      <input id="srvName" name="srvName" placeholder="IP or hostname..." type="text" />
      <label for="userName">User Name</label>
      <input id="userName" name="usrName" placeholder="username..." type="text" />
      <label for="userPass">Password</label>
      <input id="userPass" name="usrPass" placeholder="password..." type="password" />
      <input type="submit" value="Login" />
    </form>
</div>
{% if message %}
      <div class="warning">
        {{ message }}
      </div>
{% endif %}
  </body>
</html>


File: adapter.html


<html>
  <head>
    <link href="/static/styles.css" rel="stylesheet" type="text/css"></link>
    <link href="static/favicon.ico" rel="shortcut icon"></link>
    <script src="https://code.jquery.com/jquery-3.4.1.js"></script>
    <title>Adapter Info</title>
  </head>
  <body>
  <form class="frmAdapter">
  <div>
    <label class="Adapter" form="frmAdapter">Select an adapter</label>
    <select class="Adapter" id="selAdapter" name="selAdapter">
      {% for idx1 in range(data['adapter-kind']|count) %}
        <option idx1="" value="{{">{{ data['adapter-kind'][idx1]['name'] }}</option>
      {% endfor %}
    </select><noscript><input type="submit" value="Go" name=submit1></noscript>
  </div>
</form>
<form action="" class="frmResKnd" method="post">
  <div>
    <label form="frmResourceKind">Select a resource kind</label>
    <select class="ResourceKind" id="selResourceKind" name="selResourceKind">
    </select><noscript><input type="submit" value="Go" name=submit2></noscript>
    <input id="adapterURL" name="adapterURL" type="hidden" />
    <input id="adapterName" name="adapterName" type="hidden" />
    <input id="resourceKindName" name="resourceKindName" type="hidden" />
    <input type="submit" value="Retrieve Resources" />
  </div>
</form>
<script>
  $( ".Adapter" )
    .change(function () {
    lstRes = document.getElementById("selResourceKind");
    jData = {{ data|tojson }};
    x = selAdapter.value;
    $(".ResourceKind").empty();
    adapterURL.value=jData['adapter-kind'][x]['links'][1]['href'];
    adapterName.value=jData['adapter-kind'][x]['key'];
  
    for (var y=0; y < jData['adapter-kind'][x]['resourceKinds'].length; y++) {
      lstRes.options[y] = new Option(jData['adapter-kind'][x]['resourceKinds'][y]);
    }
  
    })
    .change();

  $(document).ready(function(){
    $("form").submit(function(){
      lstRes = document.getElementById("selResourceKind");
      jData = {{ data|tojson }};
      x=$("#selAdapter").prop("selectedIndex");
      y=$("#selResourceKind").prop("selectedIndex");
      resourceKindName.value=jData['adapter-kind'][x]['resourceKinds'][y];
    });
  });

  </script>

  </body>
</html>


File: resources.html

<html>
  <head>
    <link href="/static/styles.css" rel="stylesheet" type="text/css"></link>
    <link href="static/favicon.ico" rel="shortcut icon"></link>
    <script src="https://code.jquery.com/jquery-3.4.1.js"></script>
    <title>Resource Data</title>
  </head>

  <body>
  <form action="" class="frmResources" method="post">
  <div>
    <label form="frmResources">Select a resource</label>

    <select class="Resources" id="selResource" name="selResource">
      {% for idx1 in range(data['resourceList']|count) %}
        <option data="" href="" idx1="" links="" resourcelist="" value="{{">{{ data['resourceList'][idx1]['resourceKey']['name'] }}</option>
      {% endfor %}
    </select><noscript><input type="submit" value="Go" name=submit1></noscript>
    <input id="resourceID" name="resourceID" type="hidden" />
    <input type="submit" value="Get selected resource properties" />
  </div>
</form>
<script>
  $( ".Resources" )
    .change(function() {
    resourceID.value=selResource.value;
    })
    .change();
    
  $(document).ready(function(){
    $("form").submit(function(){
      resourceID.value=selResource.value;
    });
  });
  </script>

  </body>
</html>


File: properties.html


<html>
  <head>
    <link href="/static/styles.css" rel="stylesheet" type="text/css"></link>
    <link href="static/favicon.ico" rel="shortcut icon"></link>
    <script src="https://code.jquery.com/jquery-3.4.1.js"></script>
    <title>Properties</title>
  </head>

  <body>
    <table id="Properties">
      <thead>
        <th>Property</th>
        <th>Value</th>
      </thead>
      <tbody>
        {% for idx1 in range(data['property']|count) %}
<tr>
            <td>{{ data['property'][idx1]['name'] }}</td>
            <td>{{ data['property'][idx1]['value'] }}</td>
          </tr>
{% endfor %}
      </tbody>
    </table>
<div>
    <p>Edit the current resource's custom properties</p>
    <form action="" method="post">
      <label>Enter Custom Property Name</label>
      <input id="customName" name="customName" type="text" />
      <label>Enter Custom Propery Value</label>
      <input id="customValue" name="customValue" type="text" />
      <label>I understand that Property Names cannot be modified or deleted once submitted  </label>
      <label class="switch">
        <input id="chkbox" name="chkbox" type="checkbox" />
        <span class="slider round"></span>
      </label>

      <input disabled="" for="chkRisk" id="btnAgree" type="submit" value="Submit" />
    </form>
</div>
{% if message %}
      <div class="warning">
        {{ message }}
      </div>
{% endif %}

  <script>
  $('input[type="checkbox"]').click(function(){
    if(chkbox.checked){
      document.getElementById('btnAgree').disabled=false;
    }
    else {
      document.getElementById('btnAgree').disabled=true;
    }
  });

  </script>

  </body>
</html>


File: styles.css 

* {
  font-family: "Trebuchet MS", Arial, Helvetica, sans-serif;
  font-size: 12px;
}

p {
  font-size: 16px;
  color: #1ca922;
}

input[type=text], input[type=password], select {
  width: 50%;
  padding: 12px 10px;
  margin: 8px 0;
  display: inline-block;
  border: 1px solid #ccc;
  border-radius: 4px;
  box-sizing: border-box;
}

input[type=submit],button {
  width: 50%;
  background-color: #1ca922;
  color: white;
  padding: 14px 20px;
  margin: 8px 0;
  border: none;
  border-radius: 4px;
  cursor: pointer;
}

input[type=submit]:hover {
  background-color: #45a049;
}

div {
  border-radius: 5px;
  background-color: #56cc5b;
  padding: 20px;
}

table {
  border-collapse: collapse;
}

th {
  background-color: #56cc5b;
  color: white;
}

tr:nth-child(even){
  background-color: #f2f2f2;
}

tr:nth-child(odd){
  background-color: #f2f2ff;
}

table,th,td,tr {
  height: 40px;
  border: 1px solid black;
}

.warning {
  background-color: red;
  color: white;
  margin-top: 6px;
  font-size: x-large;
}

.container {
  border-radius: 5px;
  background-color: #f2f2f2;
  padding: 20px;
}

.col-25 {
  float: left;
  width: 25%;
  margin-top: 6px;
}

.col-75 {
  float: left;
  width: 75%;
  margin-top: 6px;
}

/* Clear floats after the columns */
.row:after {
  content: "";
  display: table;
  clear: both;
}

.switch {
  position: relative;
  display: inline-block;
  width: 60px;
  height: 24px;
}

/* Hide default HTML checkbox */
.switch input {
  opacity: 0;
  width: 0;
  height: 0;
}

/* The slider */
.slider {
  position: absolute;
  cursor: pointer;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  background-color: #ccc;
  -webkit-transition: .4s;
  transition: .4s;
}

.slider:before {
  position: absolute;
  content: "";
  height: 16px;
  width: 16px;
  left: 4px;
  bottom: 4px;
  background-color: white;
  -webkit-transition: .4s;
  transition: .4s;
}

input:checked + .slider {
  background-color: #1ca922;
}

input:focus + .slider {
  box-shadow: 0 0 1px #2196F3;
}

input:checked + .slider:before {
  -webkit-transform: translateX(35px);
  -ms-transform: translateX(35px);
  transform: translateX(35px);
}

/* Rounded sliders */
.slider.round {
  border-radius: 32px;
}

.slider.round:before {
  border-radius: 50%;
}

@media screen and (max-width: 600px) {
  .col-25, .col-75, input[type=submit] {
    width: 100%;
    margin-top: 0;
  }
}