Commit graph

59 commits

Author SHA1 Message Date
Dag Wieers
ee29ba5d4f plugin_formatter.py: Improve the output when processing docs (#46541)
* Improve the output when processing files

* Update docs/bin/plugin_formatter.py

Co-Authored-By: dagwieers <dag@wieers.com>

* Show progress indicator.

* Don't pp.pformat() huge structures when they aren't used anyway.

This saves ~10 seconds on my machine.

* Only show ASCII spinner if stdout is a TTY.

* Fix: E722 do not use bare 'except'
2018-12-18 11:23:49 -06:00
Dag Wieers
9bc401a272 Remove older version_added information from docs (#50097) 2018-12-18 10:50:07 -06:00
Toshio Kuratomi
3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
2018-12-16 15:03:19 -08:00
Dag Wieers
baf0ad2309 Docs: Add a "seealso" section to the module docs (#45949)
* Docs: Add a separate  "seealso" section to the module docs
to list related modules and/or related references. This clears up the notes
section for things that are actual notes.

So you can add a section in your module documentation and four types of
references are possible.

    seealso:

    # Reference by module name
    - module: aci_tenant

    # Reference by module name, including description
    - module: aci_tenant
      description: ACI module to create tenants on a Cisco ACI fabric.

    # Reference by rST documentation anchor
    - ref: aci_guide
      description: Detailed information on how to manage your ACI infrastructure using Ansible.

    # Reference by Internet resource
    - name: APIC Management Information Model reference
      description: Complete reference of the APIC object model.
      link: https://developer.cisco.com/docs/apic-mim-ref/

This PR also includes:

- Implements ansible-doc support
- Implements schema support for the seealso options
- Updates to the development documentation
- Rename filter convert_symbols_to_format to rst_ify, cfr the existing html_ify and tty_ify filters
  - This makes the existing template a lot easier to read and fixes the confusion I had myself rereading the template (again).
- We fixed the possible suboption types (which was limited to 'bool' only)

* Use latest stable instead of devel docs
2018-12-12 14:19:58 -06:00
Evgeni Golov
32dbb99bb8 plugin_formatter.py: accept multiple template dirs (#49765)
this allows to override certain templates without copying the whole
template directory
2018-12-11 16:19:40 -05:00
Sandra McCann
db3d920cfd WIP: fix build errors due to missing certified_supported label (#46835)
* fix build errors due to missing certified_supported label

* fixed references and toc
2018-10-14 19:08:06 -05:00
Dag Wieers
771a3983d2 Docs: Keep the module index clean (#46155)
A lot of modules have a short_description with a trailing dot even
though we don't want trailing dots in the index. This change removes
it when creating the document index.
2018-09-27 13:25:38 -05:00
Dag Wieers
310b0a2521 Docs: Make docsite rebuilds smarter/faster (#45062)
* Make the following scripts  idempotent so that we only have to rebuild changed docs, not all docs:

  * plugin_formatter
  * generate_man
  * dump_keywords.py
  * dump_config.py
  * testing_formatter.sh
2018-09-12 13:50:36 -07:00
Toshio Kuratomi
0873d46617 Fix ansible-doc and docsite generation for removed modules
* Fix ansible-doc wrt removed modules
  * Fix listing of modules ia ansible-doc to not complain about removed modules
    Removed modules are marked as such in the metadata but nowhere else.
    Need to retrieve the metadata when a module doesn't have a doc so that
    we can tell if it falls under this case.
  * omit removed modules from json dump
  * Print an error that the module has been removed if attempting to run
    ansible-doc on that specific module

* Get plugin_formatter to stop outputting removed modules
2018-08-24 15:37:13 -07:00
Zhikang Zhang
cda3b53035
make doc templates not case sensitive for the default value (#41158) 2018-08-13 16:47:12 -04:00
Felix Fontein
0752dc12b7 Documentation: show non-string non-iterable defaults for choices (#40212)
* Also marking non-string defaults.

* Adding list filter from #37517 to plugin_formatter.

* Simplifying list test.

* Redistribute imports
2018-06-27 22:31:47 -04:00
Toshio Kuratomi
ad2e8dd6d8 Changes to support building docs with old jinja2
This commit: fa5c0282a4 relied upon
features present in Jinja-2.10 and above.  The changes here allow us to
build the *rst* with older versions of jinja2.
2018-06-25 15:53:56 -07:00
Felix Fontein
2a29b2ff7f Make alias autodetection for symlinked modules independent of glob() order. (#40293) 2018-05-24 22:48:59 -05:00
Felix Fontein
f16933492d Fix problems in documentation generation (#40050)
* Treat C(...) as inline literal (as opposed to interpreted text).

* Making test for true and false more precise, to avoid matching 1, 1.0, etc.

* The 'is sameas' test already takes care of definedness.
2018-05-15 16:19:04 +02:00
Brian Coca
19fee0ef41 fix keyword doc generation
* use aliases when they exist
  * fix hardcoded loop attributes handling
2018-04-30 14:47:09 -04:00
Matt Clay
9e8889bb70
Fix more docs errors. (#39051)
* Fix remaining unknown-document docs errors.
* Fix last toc-tree-missing-document docs error.
2018-04-20 00:24:47 -07:00
Toshio Kuratomi
6ddc64bc7c Fixes for multiline doc descriotions breaking rst formatting
* strip whitespace to preserve indent level
* Make sure to indent subsequent lines of indentation
2018-04-19 11:37:37 -07:00
Toshio Kuratomi
8cdd75a09f Some more fixes for the docs :ref: disambiguation
The big one is that we needed to set plugin_type when we processed the by_support template.

Also added to list_of_CATEGORY_plugins page (which might not be used)
and corrected a place where I did module_name instead of name_module
2018-04-18 10:06:01 -07:00
Toshio Kuratomi
f9d83944ec regex for finding :ref: labels was flawed
The regex to find labels inside of angle brackets was not detecting
anything before so I missed all of those.
* Fixed the regex in find-plugin-refs to take care of that.
* Fixed plugin_formatter to emit namespaced labels for the M() macro.
2018-04-18 08:43:34 -07:00
Toshio Kuratomi
a1b148d190 Fix shebang so sanity tests pass 2018-04-17 15:01:29 -07:00
Toshio Kuratomi
1f0cc54ec7 fixup style problems in new script 2018-04-17 15:01:29 -07:00
Toshio Kuratomi
6b1e6b8460 Add a script for finding bare plugin :ref:s
We're porting autogenerated plugin docs so that :ref: targets are
namespaced by plugin type.  To do this effectively, we need to know
where all the :ref: targets which refer to the bare plugin names are in
the rst files.  This script will find those.
2018-04-17 15:01:29 -07:00
Joseph Herlant
00a7ff7974 Move man pages generations to rst2man (#37861) 2018-03-26 16:28:28 -07:00
Brian Coca
06b70f1614 Revert "Add a few Jinja2 tests to simplify template (#37514)"
This reverts commit b8e07f0d6e.
2018-03-20 18:20:43 -04:00
Dag Wieers
b8e07f0d6e
Add a few Jinja2 tests to simplify template (#37514)
Add list test to simplify template
2018-03-20 23:08:55 +01:00
scottb
381359a8f8
Doc build warning/broken link clean-a-palooza (#37382)
* Doc build warning/broken link clean-a-palooza, WIP commit 1.

* Fixed broken anchor

* Fixing additional broken links; converting from doc to ref.

* Fix anchor
2018-03-14 12:44:21 -07:00
John R Barker
0a2dda2060
Support <hr/> in module docs (#37030) 2018-03-05 19:18:24 +00:00
John R Barker
985f09270d
Ability to link to other pages from plugin docs (#37009)
Support relative links
2018-03-05 15:16:58 +00:00
Dag Wieers
50e989f906 docs: Workaround for non-string values (#36720)
* Workaround for non-string values

So I think the proper fix should go into html_ify, which should convert
any value into a string, rather than expecting strings only.

* My preferred solution
2018-02-27 18:28:47 +00:00
Dag Wieers
7cf08e9986
Improve parameter required values and ref fix (#36680)
This PR includes:
- An improvement to the parameter listing, where instead of yes/no, it
  is indicated with required/optional (easier when scrolling through a
  long list of parameters)
- Ensure that module reference, eg. M(foobar) do not include the module
  document title
2018-02-24 22:36:08 +01:00
Dag Wieers
8a040415f9
Fix module references in module docs (#36674) 2018-02-24 17:16:24 +01:00
Dag Wieers
ba370b178d
docsite: Add 'Edit on GitHub' for module docs (#36667)
This is something I always wanted, a 'Edit on GitHub' button for module
documentation.

I also removed the additional statement in the footer with instructions
on how to edit the module documentation.

PS The links go directly into the GitHub file editor now !
2018-02-24 03:57:37 +01:00
Brian Coca
bbd6b8bb42 Temporary (#31677)
* allow shells to have per host options, remote_tmp

added language to shell
removed module lang setting from general as  plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e

- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action

add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control

* Mostly revert expand_user as it's not quite working.

This was an additional feature anyhow.

Kept the use of pwd as a fallback but moved it to a second ssh
connection.  This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.

(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)

* fixed script and other action plugins

ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
2018-01-15 21:15:04 -08:00
scottb
f8f34c7bab
Moved generated module RSTs to their own directory. (#33966)
* Moved generated module RSTs to their own directory.

* WIP commit - fixed conflict with conf.py exclude_patterns by renaming module RST output directory to 'module_docs'.

* Added new directory to formatter module links; aded new module directory to makefile clean.

* Removed illegal comment from block.
2017-12-21 01:47:39 -08:00
John R Barker
c50da48049
Fix various RST warnings (#34084)
* Fix various RST warnings

* shorter lines
2017-12-20 15:20:05 +00:00
Matt Martz
84b8f674a7
Fix py3 docs build (#33345)
* Open file in binary mode to support py3 writes using bytes

* Update sort filter to use an attribute, so that py3 can sort dictionaries
2017-11-28 12:05:19 -06:00
Toshio Kuratomi
e07cbb033f Keywords docs (#32807)
* Fixup keyword dumping

* Clarify introductory text
* Turn links in the keyword description into seealso entries in the rst.

* Have plugin_formatter cleanup trailing whitespace

The indent filter in jinja2 < 2.10 indents blank lines by default which
leads to trailing whitespace.  Cleanup after that filter.

* Edits

* Copy edit
2017-11-10 16:59:26 -08:00
Brian Coca
2ed46e04f4 more updates to plugin/config generation (#30787)
* fixed module generation

added missing lookup page
point to plugins when plugins
made modules singular
add display for verbose an debug messages
nicer templating, changed generation order for ref
corrected links
moved most of lookup docs to plugin section

* Copy edits
* Fixed typos
* Clarified wording
2017-10-11 00:15:25 -04:00
Brian Coca
d84df2405d move from with_<lookup>: to loop:
- old functionality is still available direct lookup use, the following are equivalent

  with_nested: [[1,2,3], ['a','b','c']]

  loop: "{{lookup('nested', [1,2,3], ['a','b','c'])}}"

- avoid squashing with 'loop:'
- fixed test to use new intenal attributes
- removed most of 'lookup docs' as these now reside in the plugins
2017-10-10 15:43:49 -04:00
Brian Coca
8d56064ce4 fixed typo in if, restore module docs 2017-09-24 10:18:21 -04:00
Brian Coca
b233f3f296 updated plugin docs (#30490)
* updated  docs

- for devs:
  - added inventory/vars section
  - made some updates to general section and other plugin types
- for users:
 - added 'user' plugin section to start describing the plugins
 - docs on types, what they are and how to use

- removed ref to deleted AUTHORS file
- corrected several typos/headers
- added descriptions to config.rst template
- ignore generated files for cli/plugins and config
- remove new generated files on `make clean`
- moved details from devguid and intro doc to plugin specific pages
- pretied up lookup notes
- changed precedence ref to not conflict config
- removed duplicate config data, as config is autogenerated and up to date
- put new plugins under playbooks
- added `pass` cause rst/python dislikes fractions
- removed dupe in .gitignore, alpha sorted to avoid moar dupes
- added try cause rst/python freaks out

* generate plugins into their own dir

only do plugins that support docs
use toctree from main plugins page
2017-09-22 23:19:50 -04:00
jborean93
0e22afef52 fix for make webdocs 2017-09-20 22:45:24 -04:00
Austin S. Hemmelgarn
d5ae9d1018 Fix build on Python 3.x by using sys.maxsize. (#30424)
As outlined in https://docs.python.org/3.1/whatsnew/3.0.html#integers,
sys.maxint doesn't exist anymore in Python 3.x because there is no maximum
value for integers in Python 3.x.  sys.maxsize is present in all
versions of Python that are currently supported by Ansible, so use that
instead as an arbitrarily large index value.

Fixes the following build error when building with Python 3.x:
make -j1 docs
mkdir -p ./docs/man/man1/ ; \
PYTHONPATH=./lib docs/bin/generate_man.py --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
Traceback (most recent call last):
  File "docs/bin/generate_man.py", line 253, in <module>
    allvars[cli_name] = opts_docs(cli_class_name, cli_name)
  File "docs/bin/generate_man.py", line 119, in opts_docs
    'long_desc': trim_docstring(cli.__doc__),
  File "docs/bin/generate_man.py", line 34, in trim_docstring
    indent = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'
make: *** [Makefile:347: generate_asciidoc] Error 1
2017-09-19 16:23:16 -04:00
Adrian Likins
da15cf1f54 Generate plugin rst (#28901)
Generate rst docs for plugins 

Based on rst generated for modules. But generated plugin
docs go into docs/docsite/rst/plugins/$PLUGIN_TYPE/plugin_name.rst
( docs/docsite/rst/plugins/connection/ssh.py for ex)

* move plugins docs to rst/*_plugins/ subdirs for namespace
* Only gen support pages for modules for now.
* Add generated plugin docs to gitignore* add list_*_plugins templates
* support MODULES/PLUGINS filters for make htmldocs

   Add a 'PLUGINS=ssh' filter env var like MODULES to filter plugins to build docs for.

* fixup 'historical' version_added, skip plugins/loader.py
* Fix plugins_by_support ref link to new plugins/*/ location
* use :ref: for common_return_values, allow empty version_added
* warnings on missing doc info
* add a prefix to _random_choice
  It was colliding with the target for random_choice plugin
2017-09-19 11:14:27 -04:00
Adrian Likins
89c973445c generate rst doc pages for command line tools (#27530)
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535

* add a optparse 'desc' to lib/ansible/cli/config.py 

  The man page needs a short desc for the 'NAME' field
  which it gets from the option parse 'desc' value.

  Fixes building ansible-config man page.

* add trim_docstring from pep257 to generate_man

  use pep258 docstring trim function to fix up any indention
  weirdness inherit to doc strings (ie, lines other than
  first line being indented.

* Add refs to cli command actions

To reference ansible-vaults --vault-id option, use:

:option:`The link text here <ansible-vault --vault-id>`

or:

:option:`--vault-id <ansible-vault --vault-id>`

To reference ansible-vault's 'encrypt' action, use:

:ref:`The link text here <ansible_vault_encrypt>`

or most of the time:

:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
2017-09-07 15:44:20 -04:00
Adrian Likins
8035e68d44 Generate a rst for config and env options from base.yml (#28739)
* wip, gen docs from config/base.yml

* wip

* dont change conf.py here

* cleanup, add dump_config --template-file cli opt

* some desc are string, some are lists...

TODO: fix base.yml so it is consistent

* Filter out TODO and empty descriptions
2017-08-31 10:11:05 -04:00
Toshio Kuratomi
40ea448c7b Misc docs fixes
* Revise and link inline to the lists of modules

* Fix jinja2 objects.inv fallback path

* Fix bolding of deprecation marker

* Change module_support to link to lists via :doc:
  That links to the top of the page instead of a section.

* Add a short text for each list of maintained modules
* Change maintenance info to only display on core and network modules
2017-08-22 11:52:30 -07:00
Adrian Likins
2941e86671 Fix plugin_formatter.py -l / MODULES=none make webdocs
Fix the get_module_info check against the limit_to_modules
list so building a subset/none of the module docs works again.
2017-08-18 10:58:16 -04:00
Toshio Kuratomi
546187a8af Revamp the plugin_formatter doc generator
* Use a template to generate the category lists
* Refactor so that we first extract all of the data that we need to
  build the docs and then give that data to the templates to build with
* Add docs page listing modules ordered by support level
2017-08-15 23:12:08 -07:00
Toshio Kuratomi
af2073d057 metadata 1.1
* Add network value to support_by field.
* New support_by value, certified
* Deprecate curated in favor of certified
* Add conversion from 1.0 to 1.1 to metadata-tool
* Add supported by Red Hat field to ansible-doc output
2017-08-15 23:12:08 -07:00