| Age | Commit message (Collapse) | Author |
|---|
|
Due to the quite messy heritage of this codebase -- including a large part
of it being just a fork of my earlier personal project ZCL -- there were
many different conventions for how to pretty-print and diff values in the
tests in different parts of the codebase.
To reduce the dependency sprawl, this commit now standardizes on:
- github.com/davecgh/go-spew for pretty-printing
- github.com/google/go-cmp for diffing
These two dependencies were already present anyway, are the most general
out of all of the candidates, and are also already in use by at least some
of HCL's most significant callers, such as HashiCorp Terraform.
The version of go-cmp we were previously using seems to have a bug that
causes the tests to crash when run under the Go race detector, so I've
also upgraded that dependency to latest here to clear that bug.
|
|
|
|
|
|
|
|
|
|
|
|
To match with the Unicode support in Go 1.21, we'll now use the Unicode 15
tables when we're normalizing Unicode strings and when counting
user-perceived characters ("grapheme clusters") for source position
purposes.
|
|
* [COMPLIANCE] Add Copyright and License Headers
* add copywrite file and revert headers in testdata
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
|
|
hclwrite: fix data race in formatSpaces()
|
|
Port fuzz testing to Go 1.18 native fuzzing
|
|
This global variable is supposed to be treated as immutable.
|
|
|
|
* Fix data race in `formatSpaces()` by inlining shared state.
* Run tests with the race detector enabled.
Signed-off-by: Ryan Cragun <me@ryan.ec>
|
|
This test is aiming to verify that the tokens produced by TokensForValue
when given a map or object value will stay consistent with the tokens
produced by the lower-level TokensForObject function under future
maintenence.
However, since cty object types and maps don't preserve attribute/element
order, the result from TokensFromValue is always a lexical sort of the
attribute names or keys. TokensForObject is intentionally more flexible
by allowing the caller to control the order of the generated attributes,
but that means that we expect consistency only if the caller provides the
attributes in lexical order.
This test therefore now provides the TokensForObject argument in the
expected order to make this test valid. Previously one of the test cases
would fail 50% of the time due to the attributes being written out in
reverse order.
|
|
The "TokensFor..." family of functions all aim to construct valid
raw token sequences representing particular syntax constructs.
Previously we had only "leaf" functions TokensForValue and
TokensForTraversal, but nothing to help with constructing compound
structures.
Here we add TokensForTuple, TokensForObject, and
TokensForFunctionCall which together cover all of the constructs
that HCL allows static analysis of, and thus constructs where it's
likely that someone would want to generate an expression that
is interpreted purely by its syntax and not resolved into a value.
What all of these have in common is that they take other Tokens
values as arguments and include them verbatim as part of their
result, with the caller being responsible for making sure these
smaller units are themselves valid expression tokens.
This also adds TokensForIdentifier as a convenient shorthand for
generating single-identifier tokens, which is particularly useful
for populating the attribute names in TokensForObject.
|
|
HCL uses a number of upstream libraries that implement algorithms defined
in Unicode. This commit is updating those libraries all to versions that
have Unicode 13 support.
The main implication of this for HCL directly is that when it returns
column numbers in source locations it will count characters using the
Unicode 13 definition of "character", which includes various new
multi-codeunit characters added in Unicode 13.
These new version dependencies will also make Unicode 13 support available
for other functionality that HCL callers might use, such as the stdlib
functions in upstream cty, even though HCL itself does not directly use
those.
|
|
|
|
So far the expression parentheses syntax has been handled entirely in the
parser and has been totally invisible in the AST. That's fine for typical
expression evaluation, but over the years it's led to a few quirky
behaviors in less common situations where we've assumed that all
expressions are covered by the AST itself or by the source ranges that the
AST captures.
In particular, hclwrite assumes that all expressions will have source
ranges that cover their tokens, and it generates an incorrect physical
syntax tree when the AST doesn't uphold that.
After resisting through a few other similar bugs, this commit finally
introduces an explicit AST node for parentheses, which makes the
parentheses explicit in the AST and captures the larger source range that
includes the TokenOParen and the TokenCParen.
This means that parentheses will now be visible as a distinct node when
walking the AST, as reflected in the updated tests here. That may cause
downstream applications that traverse the tree to exhibit different
behaviors but we're not considering that as a "breaking change" because
the Walk function doesn't make any guarantees about the specific AST
shape.
|
|
The hclsyntax package permits block labels to be blank quoted strings,
and defers to the application to determine if this is valid or not. This
commit updates hclwrite to allow the same behaviour.
This is in response to an upstream bug report on Terraform, which uses
hclwrite to implement its fmt subcommand. Given a block with a blank
quoted string label, it currently deletes the label when formatting.
This is inconsistent with Terraform's behaviour when parsing HCL, which
treats empty labels differently from missing labels.
|
|
All of the other subdivisions of a block were already nodes, but we'd
represented the labels as an undifferentiated set of nodes belonging
directly to the block's child node list.
Now that we support replacing the labels in the public API, that's a good
excuse to refactor this slightly to make the labels their own node. As
well as being consistent with everything else in Block, this also makes
it easier to implement the Block.SetLabels operation because we can
just change the children of the labels node, rather than having to
carefully identify and extract the individual child nodes of the block
that happen to represent labels.
Internally this models the labels in a similar sort of way as the content
of a body, although we've kept the public API directly on the Block type
here because that's a more straightforward model for the use-cases we
currently know and matches better with the API of hcl.Block. This is just
an internal change for consistency.
I also added a few tests for having comments interspersed with labels
while I was here, because that helped to better exercise the new
parseBlockLabels function.
|
|
While implementing Block.SetLabels(), I found a new hclwrite parser bug.
The NewBlock() method records positions of TokenOBrace / TokenCBrace.
Nevertheless when generating blocks via hclwrite.ParseConfig(),
they were not recorded.
The position of TokenOBrace is needed for Block.SetLabels(),
so I also fixed this existing bug.
|
|
Fixes #338
Add methods to update block type and labels to enable us to refactor HCL
configurations such as renaming Terraform resources.
- `*Block.SetType(typeName string)`
- `*Block.SetLabels(labels []string)`
Some additional notes about SetLabels:
Since we cannot assume that old and new labels are equal in length,
remove old labels and insert new ones before TokenOBrace.
To implement this, I also added the following methods.
- `*nodes.Insert(pos *node, c nodeContent) *node`
- `*nodes.InsertNode(pos *node, n *node) *node`
They are similar to the existing Append / AppendNode,
but insert a node before a given position.
|
|
Fix typos in hclwrite/parser.go
|
|
|
|
The following expression caused a panic in hclwrite:
a = foo.*
This was due to the unusual dotted form of a full splat (where the splat
operator is at the end of the expression) being generated with an
invalid source range. In the full splat case, the end of the range was
uninitialized, which caused the token slice to be empty, and thus the
panic.
This commit fixes the bug, adds test coverage, and includes some bonus
tests for other splat expression cases.
|
|
hclwrite: handle legacy dot access of numeric indexes
|
|
The previous syntax for object and map values was a single line of
key-value pairs. For example:
object = { bar = 5, baz = true, foo = "foo" }
This is very compact, but in practice for many HCL values, less readable
and less common than a multi-line object format. This commit changes the
generated output from hclwrite to one line per attribute.
Examples of the new format:
// Empty object/map is a single line
a = {}
// Single-value object/map has the attribute on a separate line
b = {
bar = 5
}
// Multi-value object/map has one line per attribute
c = {
bar = 5
baz = true
}
|
|
|
|
|
|
HCL uses grapheme cluster segmentation to produce accurate "column"
indications in diagnostic messages and other human-oriented source
location information. Each new major version of Unicode introduces new
codepoints, some of which are defined to combine with other codepoints to
produce a single visible character (grapheme cluster).
We were previously using the rules from Unicode 9.0.0. This change
switches to using the segmentation rules from Unicode 12.0.0, which is
the latest version at the time of this commit and is also the version of
Unicode used for other purposes by the Go 1.14 runtime.
HCL does not use text segmentation results for any purpose that would
affect the meaning of decoded data extracted from HCL files, so this
change will only affect the human-oriented source positions generated for
files containing characters that were newly-introduced in Unicode 10, 11,
or 12. (Machine-oriented uses of source location information are based on
byte offsets and not affected by text segmentation.)
|
|
Some HCL callers make the (reasonable) assumption that the overall source
range of an expression will be a superset of all of the ranges of its
child expressions, for purposes such as extraction of source code
snippets, parse tree annotation in hclwrite, text editor analysis
functions like "go to reference", etc.
The IndexExpr type was not previously honoring that assumption, since its
source range was placed around only the bracket portion. That is a good
region to use when reporting errors relating to the index operation, but
it is not a faithful representation of the full extent of the expression.
In order to meet both of these requirements at once, IndexExpr now has
both SrcRange covering the entire expression and BracketRange covering
the index part delimited by brackets. We can then use BracketRange in
our error messages but return SrcRange as the result of the general
Range method that is common to all expression types.
|
|
We currently have functions for constructing new expressions from either
constant values or from traversals, but some surgical updates require
producing a more complex expression.
In the long run perhaps we'll have some mechanism for constructing valid
expressions via a high-level AST-like API, similar to what we already have
for structural constructs, but as a simpler first step here we add a
mechanism to just write raw tokens directly into an expression, with the
caller being responsible for making sure those tokens represent valid
HCL expression syntax.
Since this new API treats the given tokens as unstructured, the resulting
expression can't fully support the whole of the expression API, but it's
good enough for writing in complex expressions without disturbing existing
content elsewhere in the input file.
|
|
|
|
|
|
Fixes #310
The type of `Body.items` is a `nodeSet` (That is, `map[*node]struct{}`),
so the order of the list returned by Body.Blocks () was unstable.
|
|
Previously we allowed adding both attributes and blocks, and we allowed
updating attributes, but we had no mechanism to surgically remove
attributes and blocks altogether.
|
|
|
|
The main HCL package is more visible this way, and so it's easier than
having to pick it out from dozens of other package directories.
|
|
This is in preparation for the first v2 release from the main HCL
repository.
|
|
|
|
|
|
Waiting for TokenCHeredoc never ends since scanTokens() does not
produce
TokenNewlines inside heredocs.
Related Issue: hashicorp/terraform#21434
|
|
|
|
|
|
|
|
closes #65
- add missing edge case for two interpolated items next to each other
- add tests for both quote and heredoc cases
|
|
The symmetrical spaces around colons in conditionals are important for
familiarity with C-like languages, so we'll instead accept spaces around
colons in our HCL2-unique "for expression" construct.
|
|
Leading whitespace is significant in heredocs, so we'll avoid making any
indentation adjustments for lines between OHeredoc and CHeredoc.
This fixes #31.
|
|
Our normal ruleset thinks that the "in" keyword here is a variable
reference and so writes it as "in[y]". Since there's never any reason for
a variable to appear immediately after another variable, we can check
for a preceding identifier as a heuristic to recognize whether in is
probably being used as a keyword rather than as a variable.
This is not exact, but the only time this should be a false positive is
if there were a syntax error in the input, and we don't make any
guarantees about the result in that case anyway.
This fixes #52.
|
|
This relaxes our previous spec to include a special form from HCL 1:
foo { bar = baz }
Although we normally require each argument to be on a line of its own, as
a special case we allow a block to be defined with a single nested
argument all on one line.
Only one nested argument definition is allowed, and a nested block
definition like "foo { bar {} }" is also disallowed in order to force the
more-readable split of bar {} onto a line of its own.
This is a pragmatic addition for broader compatibility with HCL 1-oriented
input. This single-line usage is not considered idiomatic HCL 2 and may
in future be undone by the formatter, though for now it is left as-is
aside from the spacing around the braces.
This also changes the behavior of the source code formatter to include
spaces on both sides of braces. This mimicks the formatting behavior of
HCL 1 for this situation, and (subjectively) reads better even for other
one-line braced expressions like object constructors and object for
expressions.
|
