wbrawner/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

New doc guide for GCP (#41753)

Browse source 
* New doc guide for GCP

* Changes per PR

* Addressing PR comments on GCE scenario guide
This commit is contained in:
Alex Stephen 2018-07-06 07:09:13 -07:00 committed by Alicia Cozine
parent 7ee86ae370
commit 2916adc88a
1 changed files with 155 additions and 186 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

341
docs/docsite/rst/scenario_guides/guide_gce.rst
View file

@ -4,220 +4,230 @@ Google Cloud Platform Guide
.. gce_intro:
Introduction
------------
--------------------------
.. note:: This section of the documentation is under construction. We are in the process of adding more examples about all of the GCE modules and how they work together. Upgrades via github pull requests are welcomed!
Ansible + Google have been working together on a set of auto-generated
Ansible modules designed to consistently and comprehensively cover the entirety
of the Google Cloud Platform.
Ansible contains modules for managing Google Compute Engine resources, including creating instances, controlling network access, working with persistent disks, and managing
load balancers. Additionally, there is an inventory plugin that can automatically suck down all of your GCE instances into Ansible dynamic inventory, and create groups by tag and other properties.
Ansible contains modules for managing Google Cloud Platform resources,
including creating instances, controlling network access, working with
persistent disks, managing load balancers, and a lot more.
The GCE modules all require the apache-libcloud module which you can install from pip:
These new modules can be found under a new consistent name scheme "gcp_*"
(Note: gcp_target_proxy and gcp_url_map are legacy modules, despite the "gcp_*"
name. Please use gcp_compute_target_proxy and gcp_compute_url_map instead).
Additionally, the gcp_compute inventory plugin can discover all GCE instances
and make them automatically available in your Ansible inventory.
You may see a collection of other GCP modules that do not conform to this
naming convention. These are the original modules primarily developed by the
Ansible community. You will find some overlapping functionality such as with
the the "gce" module and the new "gcp_compute_instance" module. Either can be
used, but you may experience issues trying to use them together.
While the community GCP modules are not going away, Google is investing effort
into the new "gcp_*" modules. Google is committed to ensuring the Ansible
community has a great experience with GCP and therefore recommends that begin
adopting these new modules if possible.
Introduction
---------------
The Google Cloud Platform (GCP) modules require both the ``requests`` and the
``google-auth`` libraries to be installed.
.. code-block:: bash
$ pip install apache-libcloud
$ pip install requests google-auth
.. note:: If you're using Ansible on macOS, libcloud also needs to access a CA cert chain. You'll need to download one (you can get one for `here <http://curl.haxx.se/docs/caextract.html>`_.)
Credentials
-----------
It's easy to create a GCP account with credentials for Ansible. You have multiple options to
get your credentials - here are two of the most common options:
To work with the GCE modules, you'll first need to get some credentials in the
* Service Accounts (Recommended): Use JSON service accounts with specific permissions.
* Machine Accounts: Use the permissions associated with the GCP Instance you're using Ansible on.
For the following examples, we'll be using service account credentials.
To work with the GCP modules, you'll first need to get some credentials in the
JSON format:
1. `Create a Service Account <https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount>`_
2. `Download JSON credentials <https://support.google.com/cloud/answer/6158849?hl=en&ref_topic=6262490#serviceaccounts>`_
There are three different ways to provide credentials to Ansible so that it can talk with Google Cloud for provisioning and configuration actions:
Once you have your credentials, there are two different ways to provide them to Ansible:
.. note:: If you would like to use JSON credentials you must have libcloud >= 0.17.0
* by providing to the modules directly
* by populating a ``secrets.py`` file
* by specifying them directly as module parameters
* by setting environment variables
Calling Modules By Passing Credentials
``````````````````````````````````````
Providing Credentials as Module Parameters
``````````````````````````````````````````
For the GCE modules you can specify the credentials as arguments:
* ``auth_kind``: type of authentication being used (choices: machineaccount, serviceaccount, application)
* ``service_account_email``: email associated with the project
* ``credentials_file``: path to the JSON credentials file
* ``project_id``: id of the project
* ``service_account_file``: path to the JSON credentials file
* ``project``: id of the project
* ``scopes``: The specific scopes that you want the actions to use.
For example, to create a new instance using the cloud module, you can use the following configuration:
For example, to create a new IP address using the `gcp_compute_address` module,
you can use the following configuration:
.. code-block:: yaml
- name: Create instance(s)
- name: Create IP address
hosts: localhost
connection: local
gather_facts: no
vars:
service_account_email: unique-id@developer.gserviceaccount.com
credentials_file: /path/to/project.json
project_id: project-id
machine_type: n1-standard-1
image: debian-7
service_account_file: /home/my_account.json
project: my-project
auth_kind: serviceaccount
scopes:
- www.googleapis.com/auth/compute
tasks:
- name: Launch instances
gce:
instance_names: dev
machine_type: "{{ machine_type }}"
image: "{{ image }}"
service_account_email: "{{ service_account_email }}"
credentials_file: "{{ credentials_file }}"
project_id: "{{ project_id }}"
- name: Allocate an IP Address
gcp_compute_address:
state: present
name: 'test-address1'
region: 'us-west1'
project: "{{ project }}"
auth_kind: "{{ auth_kind }}"
service_account_file: "{{ service_account_file }}"
scopes: "{{ scopes }}"
When running Ansible inside a GCE VM you can use the service account credentials from the local metadata server by
setting both ``service_account_email`` and ``credentials_file`` to a blank string.
Configuring Modules with secrets.py
```````````````````````````````````
Create a file ``secrets.py`` looking like following, and put it in some folder which is in your ``$PYTHONPATH``:
.. code-block:: python
GCE_PARAMS = ('i...@project.googleusercontent.com', '/path/to/project.json')
GCE_KEYWORD_PARAMS = {'project': 'project_id', 'datacenter': 'gce_zone'}
Ensure to enter the email address from the created services account and not the one from your main account.
Now the modules can be used as above, but the account information can be omitted.
If you are running Ansible from inside a GCE VM with an authorized service account you can set the email address and
credentials path as follows so that get automatically picked up:
.. code-block:: python
GCE_PARAMS = ('', '')
GCE_KEYWORD_PARAMS = {'project': 'project_id', 'datacenter': ''}
Configuring Modules with Environment Variables
Providing Credentials as Environment Variables
``````````````````````````````````````````````
Set the following environment variables before running Ansible in order to configure your credentials:
.. code-block:: bash
GCE_EMAIL
GCE_PROJECT
GCE_CREDENTIALS_FILE_PATH
GCP_AUTH_KIND
GCP_SERVICE_ACCOUNT_EMAIL
GCP_SERVICE_ACCOUNT_FILE
GCP_SCOPES
GCE Dynamic Inventory
---------------------
The best way to interact with your hosts is to use the gce inventory plugin, which dynamically queries GCE and tells Ansible what nodes can be managed.
The best way to interact with your hosts is to use the gcp_compute inventory plugin, which dynamically queries GCE and tells Ansible what nodes can be managed.
Note that when using the inventory script ``gce.py``, you also need to populate the ``gce.ini`` file that you can find in the contrib/inventory directory of the ansible checkout.
To use the gcp_compute inventory plugin, create a file that ends in .gcp.yml file in your root directory. The gcp_compute inventory script takes in the same authentication
information as any module.
To use the GCE dynamic inventory script, copy ``gce.py`` from ``contrib/inventory`` into your inventory directory and make it executable. You can specify credentials for ``gce.py`` using the ``GCE_INI_PATH`` environment variable -- the default is to look for gce.ini in the same directory as the inventory script.
Here's an example of a valid inventory file:
Let's see if inventory is working:
.. code-block:: yaml
.. code-block:: bash
plugin: gcp_compute
projects:
- google.com:graphite-playground
filters:
auth_kind: serviceaccount
service_account_file: /home/alexstephen/my_account.json
$ ./gce.py --list
You should see output describing the hosts you have, if any, running in Google Compute Engine.
Now let's see if we can use the inventory script to talk to Google.
.. code-block:: bash
$ GCE_INI_PATH=~/.gce.ini ansible all -i gce.py -m setup
hostname | success >> {
"ansible_facts": {
"ansible_all_ipv4_addresses": [
"x.x.x.x"
],
As with all dynamic inventory scripts in Ansible, you can configure the inventory path in ansible.cfg. The recommended way to use the inventory is to create an ``inventory`` directory, and place both the ``gce.py`` script and a file containing ``localhost`` in it. This can allow for cloud inventory to be used alongside local inventory (such as a physical datacenter) or machines running in different providers.
Executing ``ansible`` or ``ansible-playbook`` and specifying the ``inventory`` directory instead of an individual file will cause ansible to evaluate each file in that directory for inventory.
Let's once again use our inventory script to see if it can talk to Google Cloud:
.. code-block:: bash
$ ansible all -i inventory/ -m setup
hostname | success >> {
"ansible_facts": {
"ansible_all_ipv4_addresses": [
"x.x.x.x"
],
The output should be similar to the previous command. If you're wanting less output and just want to check for SSH connectivity, use "-m" ping instead.
Use Cases
---------
For the following use case, let's use this small shell script as a wrapper.
.. code-block:: bash
#!/usr/bin/env bash
PLAYBOOK="$1"
if [[ -z $PLAYBOOK ]]; then
echo "You need to pass a playbook as argument to this script."
exit 1
fi
export SSL_CERT_FILE=$(pwd)/cacert.pem
export ANSIBLE_HOST_KEY_CHECKING=False
if [[ ! -f "$SSL_CERT_FILE" ]]; then
curl -O http://curl.haxx.se/ca/cacert.pem
fi
ansible-playbook -v -i inventory/ "$PLAYBOOK"
Executing ``ansible-inventory --list -i <filename>.gcp.yml`` will create a list of GCP instances that are ready to be configured using Ansible.
Create an instance
``````````````````
The GCE module provides the ability to provision instances within Google Compute Engine. The provisioning task is typically performed from your Ansible control server against Google Cloud's API.
The full range of GCP modules provide the ability to create a wide variety of
GCP resources with the full support of the entire GCP API.
A playbook would looks like this:
The following playbook creates a GCE Instance. This instance relies on a GCP
network and a Disk. By creating the Disk and Network separately, we can give as
much detail as necessary about how we want the disk and network formatted. By
registering a Disk/Network to a variable, we can simply insert the variable
into the instance task. The gcp_compute_instance module will figure out the
rest.
.. code-block:: yaml
- name: Create instance(s)
- name: Create an instance
hosts: localhost
gather_facts: no
connection: local
vars:
machine_type: n1-standard-1 # default
image: debian-7
service_account_email: unique-id@developer.gserviceaccount.com
credentials_file: /path/to/project.json
project_id: project-id
project: my-project
auth_kind: serviceaccount
service_account_file: /home/my_account.json
zone: "us-central1-a"
region: "us-central1"
tasks:
- name: Launch instances
gce:
instance_names: dev
machine_type: "{{ machine_type }}"
image: "{{ image }}"
service_account_email: "{{ service_account_email }}"
credentials_file: "{{ credentials_file }}"
project_id: "{{ project_id }}"
tags: webserver
register: gce
- name: create a disk
gcp_compute_disk:
name: 'disk-instance'
size_gb: 50
source_image: 'projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts'
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
scopes:
- https://www.googleapis.com/auth/compute
state: present
register: disk
- name: create a network
gcp_compute_network:
name: 'network-instance'
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
scopes:
- https://www.googleapis.com/auth/compute
state: present
register: network
- name: create a address
gcp_compute_address:
name: 'address-instance'
region: "{{ region }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
scopes:
- https://www.googleapis.com/auth/compute
state: present
register: address
- name: create a instance
gcp_compute_instance:
state: present
name: test-vm
machine_type: n1-standard-1
disks:
- auto_delete: true
boot: true
source: "{{ disk }}"
network_interfaces:
- network: "{{ network }}"
access_configs:
- name: 'External NAT'
nat_ip: "{{ address }}"
type: 'ONE_TO_ONE_NAT'
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
scopes:
- https://www.googleapis.com/auth/compute
register: instance
- name: Wait for SSH to come up
wait_for: host={{ item.public_ip }} port=22 delay=10 timeout=60
loop: "{{ gce.instance_data }}"
wait_for: host={{ instance.address }} port=22 delay=10 timeout=60
- name: Add host to groupname
add_host: hostname={{ item.public_ip }} groupname=new_instances
loop: "{{ gce.instance_data }}"
add_host: hostname={{ instance.address }} groupname=new_instances
- name: Manage new instances
hosts: new_instances
@ -230,45 +240,4 @@ A playbook would looks like this:
Note that use of the "add_host" module above creates a temporary, in-memory group. This means that a play in the same playbook can then manage machines
in the 'new_instances' group, if so desired. Any sort of arbitrary configuration is possible at this point.
Configuring instances in a group
````````````````````````````````
All of the created instances in GCE are grouped by tag. Since this is a cloud, it's probably best to ignore hostnames and just focus on group management.
Normally we'd also use roles here, but the following example is a simple one. Here we will also use the "gce_net" module to open up access to port 80 on
these nodes.
The variables in the 'vars' section could also be kept in a 'vars_files' file or something encrypted with Ansible-vault, if you so choose. This is just
a basic example of what is possible::
- name: Setup web servers
hosts: tag_webserver
gather_facts: no
vars:
machine_type: n1-standard-1 # default
image: debian-7
service_account_email: unique-id@developer.gserviceaccount.com
credentials_file: /path/to/project.json
project_id: project-id
roles:
- name: Install lighttpd
apt: pkg=lighttpd state=installed
sudo: True
- name: Allow HTTP
local_action: gce_net
args:
fwname: "all-http"
name: "default"
allowed: "tcp:80"
state: "present"
service_account_email: "{{ service_account_email }}"
credentials_file: "{{ credentials_file }}"
project_id: "{{ project_id }}"
By pointing your browser to the IP of the server, you should see a page welcoming you.
Upgrades to this documentation are welcome, hit the github link at the top right of this page if you would like to make additions!
For more information about Google Cloud, please visit the `Google Cloud website <https://cloud.google.com>`