Vale

The goal of this page is to introduce the contributors to Vale as a tool, and how to work with as part of documentation process.

Overview

Vale is a tool written in go, which brings code-like linting to natural languages. The purpose of Vale here is to standardize the documentation process and automate the proofreading. About Vale → Vale documentation. Files which are required to understand the working of Vale in the pipeline:

docs-site
├── scripts
    ├── vale.sh
├── .vale
    ├── HPC
        ├── Acronyms.yml
        ├── ...
    ├── Microsoft
        ├── Accessbility.yml
        ├── ...
    ├── Vocab
        ├── HPC
            ├── accept.txt
            ├── reject.txt
├── .vale.ini
├── .gitlab-ci.yml

Vale in the CI Process

Whenever a merge request is created, Vale runs for each of the changed/modified files in that merge-request. It’s a part of the test job in the CI process for the automated tests. You can check, .gitlab-ci.yml file to check when it’s triggered, and scripts/vale.sh is executed in the process.

Configuration

The configuration for Vale here is controlled by the .vale.ini file in the codebase.

StylesPath = .vale
Vocab = HPC

# MinAlertLevel doesn't relate to the exit codes
MinAlertLevel = suggestion

[*.adoc]
#BasedOnStyles = Vale
BasedOnStyles = Vale,Microsoft,HPC

# Ignore SVG markup
TokenIgnores = (\*\*\{\w*\}\*\*)

Parameter

Description

StylesPath

It’s configured to use .vale directory for the styles.

Vocab

To keep custom lists of terminologies independent of the styles. Discussed in detail in the below section Vocab.

[*.adoc]

Checks all the files with the extension .adoc as the documentation files.

MinAlertLevel

Set to suggestion , thereby including the levels error, warning and suggestion.

BasedOnStyles

Specifies that the styles(Vale, Microsoft, HPC) have all their rules enabled. Discussed in detail in the next section Styles.

TokenIgnores

To ignore an inline section of text that matches the pattern *\*\{\w*\}*\*. For detailed explanation, refer to the Scoping section.

  • To enable only a certain rule in a style, you can do the following:

#Enables only this rule
HPC.Rule = YES

  • To disable individual rules or change their severity level, do the below:

BasedOnStyles = Vale,Microsoft,HPC

#Disables Rule1 from HPC style
HPC.Rule1 = NO

#Changs the severity level of Rule2 from HPC style
HPC.Rule2 = error

Styles

  • Styles are a powerful extension system provided with Vale, where you can define/extend rules of your documentation languages. Rules can be written in .yml files without any prior programming knowledge which can come in handy.

  • In the docs-site repository, .vale directory is set as the root directory for the defined styles, and the directory structure looks like the following,

docs-site
    .vale
    ├── HPC
    │   ├── Acronyms.yml
    │   ...
    ├── Microsoft
    │   ├──  Accessibility.yml
    │   ...
    └── Vocab
        ├── HPC
            ├──  accept.txt
            ├── reject.txt

  • The building blocks of styles are rules (YAML files ending in .yml), which use extension points to perform specific tasks.

  • Thus, the set of rules which are listed under the .vale/HPC/ and .vale/Microsoft/ are applied to the file changes.

  • A simple example rule is discussed below for the rule listed in .vale/HPC/Cliches.yml.

cliches.yml

---
#
# Detect and avoid using cliches
#
extends: existence
message: "Try to avoid using clichés like '%s'."
ignorecase: true
level: warning
tokens:
  - a chip off the old block
  - a clean slate

Parameter

Description

extends

The most general extension point is existence and it looks for the "existence" of particular tokens. The other extension points are substitution, occurrence, consistency, conditional, capitalization, readability, spelling, and sequence. Refer this for detailed explanation of each extension point point_right Vale Style Extensions

message

Message shown to the user when the rule is broken.

level

The severity level of this rule is warning. Other possible values are error, suggestion

ignorecase

Expects a boolean value. If true, matches all the words that are case-insensitive.

tokens

A list of values that can contain anything from simple phrases to complex regex expressions. In the above example, if Vale sees cliches like a chip off the old block or a clean slate, it will flag a warning message.

Vocab

  • This allows to keep custom lists of terminologies independent of the styles. Each Vocab is a single folder and stored in the path .vale/Styles/Vocab/<name>.

  • It consists of two files - accept.txt, reject.txt. It can contain a word, phrase or regular expression.

  • Entries in accept.txt file are added to every exception list in all styles listed in BasedOnStyles parameter in the .vale.ini configuration file. These entries are automatically added to Vale.Terms to ensure that any occurrences of these words/phrases match exactly the corresponding entry in the accept.txt file.

  • On the other hand, entries in reject.txt are automatically added to Vale.Avoid that will flag all occurrences as errors.

  • Also, multiple vocabularies can be added to the Vocab folder.

Scoping

  • AsciiDoc is supported by Vale. Hence, code blocks, image paths, and inline literals are ignored by default.

  • To ignore certain non-standard sections of text, BlockIgnores and TokenIgnores can be used.

BlockIgnores

  • To ignore entire blocks of text, you can define BlockIgnores. Consider the following code snippet:

{< output >}
Answer: 10
{< /output >}

  • To make Vale ignore the above blocks of text in your file, you can define the following BlockIgnores regex pattern and add it to the .vale.ini configuration file like below.

#Ignore blocks of text
BlockIgnores = (?s) *({< output >}.*?{< ?/ ?output >})

  • Also, you can define more than one pattern separated by commas.

TokenIgnores

  • Similarly, to ignore an inline section of the text, use TokenIgnores and define a pattern in the .vale-ini file like below.

#Ignore inline sections of text
TokenIgnores = <pattern>

  • Also, you can define more than one pattern separarted by commas.

  • To learn more and test the above regex pattern, refer to the site point_right Regex.

  • For more information about Vale scoping, check the following point_right Scoping.

Testing Locally

Suppose, you have the following changes in the below file. Assume that the file is in the path /content/discovery/modules/ROOT/pages/home/test.adoc

test.adoc :

= Testing Vale

* Slurm is an open-source workload manager/scheduler for the Discovery Cluster. Slurm is basically the intermediary between the Login nodes and compute nodes.

* So, the slurm scheduler is the gateway for the users on the login nodes to submit work/jobs to the compute nodes for processing.

* Consider the below example.

image::slurm-logo.png[Slurm]


* Docs contributed by Morkov.

Running Vale

  • To run Vale against the file test.adoc locally, run the following command from the root of the docs-site folder:

docs-site$make vale filename="content/discovery/modules/ROOT/pages/home/test.adoc"

Output:

 5:3    error       Don't start a sentence with     HPC.So
                    'So,'.
 5:11   error       Use 'Slurm' instead of          Vale.Terms
                    'slurm'.
 12:3   suggestion  Consider "documentation"        HPC.SubstitutionSuggestions
                    instead of "Docs".
 12:23  error       Did you really mean 'Morkov'?   Vale.Spelling

✖ 3 errors, 0 warnings and 1 suggestion in 1 file.

Fixing Vale Errors/Warnings

  • Regarding the error at 5:3, Vale says that the sentence shouldn’t start with So,. It can be replaced by other options like Hence which will solve the error.

  • Regarding the error at 5:11, Vale says to use Slurm instead of slurm. The reason behind the error is that, the file accept.txt has the entry Slurm. Hence, if Vale finds any occurrences of slurm, it will throw an error and asks to be replaced by Slurm. However, there is a lowercase version of slurm in the image path(image::slurm-logo.png[Slurm]) which goes undetected/ignored by the Vale. This is because Vale supports AsciiDoc and ignores the styles on code blocks, image paths and inline literals.

  • To fix the Vale.Spelling error on line 12:23, add Morkov to accept.txt file.

  • Also, replace docs with documentation to fix the suggestion.

  • Try to resolve the errors found, and look at the warnings, and suggestions too if you have time before pushing it to the remote branch and creating a merge request.

  • If the error and the rule behind it doesn’t make sense or you find it irrelevant, you can alter the rules from the .vale directory. Please be extra careful when updating the rules.

Pipeline Failure

  • Assume that the local feature branch has some changes in the .adoc files and the changes are committed without running/testing Vale locally on the file changes.

  • After the changes are pushed to the remote branch and a merge request is created, a test pipeline will get launched.

  • The pipeline runs Vale on the changed files(.adoc). If Vale detects any error, the pipeline job gets failed and the merge request can’t be merged to the master branch. However, if there are only warnings/suggestions without any errors, the pipeline will run successfully if there are no build errors.

  • Now, the Vale errors have to be fixed from your local feature branch and needs to be committed again to the remote branch. Hence, it’s highly recommended to run Vale locally for the changed .adoc files, then commit and push it to the remote branch. To test the file changes with Vale locally, refer to the above section point_right Testing Vale Locally

Writing a New Style Rule

  • Vale has some defined structure, rules on which you can extend your rules on. Here is an example on how you make your own rules with Vale. The below example shows how a new rule can be written and added to Vale.

  • Go to the path .vale/HPC and create a file using the command vi remove.yml

  • Copy and paste the below code.

# All rules should define the following header keys:
#
# `extends` indicates the extension point being used (see below for information
# on the possible values).
extends: existence
# `message` is shown to the user when the rule is broken.
#
# Many extension points accept format specifiers (%s), which are replaced by
# extracted values. See the extension-specific sections below for more details.
message: "Consider removing '%s'"
# `level` assigns the rule's severity.
#
# The accepted values are suggestion, warning, and error.
level: warning
#
# It can be a word,phrase or regex expression
tokens:
    - best

  • In the above example, if Vale detects the word best, it will throw a warning message, saying that Consider removing best.

  • For more details on the Vale rules → Vale Style Guide.

References