| Age | Commit message (Collapse) | Author |
|---|
|
We were calling .Range() on any unknown sourceVal, without first checking
whether it was marked. That method panics if called on a marked value,
so we need to strip that off first.
While testing this I found some return paths that weren't properly
transferring the source value's marks to the output, and so this also
addresses those so that all return paths preserve whatever markings are
present on the source value.
In particular, if a non-list/set/tuple value gets "upgraded" into a tuple
then we must transfer its marks onto the tuple, because the decision about
constructing that value was based on characteristics of the source value.
|
|
for namespaced functions
|
|
|
|
This introduces a new syntax which allows function names to have namespace
prefixes, with the different name parts separated by a double-colon "::"
as is common in various other C-derived languages which need to
distinguish between scope resolution and attribute/field traversal.
Because HCL has separate namespaces for functions and variables, we need
to use different punctuation for each to avoid creating parsing ambiguity
that could be resolved only with infinite lookahead.
We cannot retroactively change the representation of function names to be
a slice of names without breaking the existing API, and so we instead
adopt a convention of packing the multi-part names into single strings
which the parser guarantees will always be a series of valid identifiers
separated by the literal "::" sequence. That means that applications will
make namespaced functions available in the EvalContext by naming them in
a way that matches this convention.
This is still a subtle compatibility break for any implementation of the
syntax-agnostic HCL API against another syntax, because it may now
encounter function names in the function table that are not entirely
valid identifiers. However, that's okay in practice because a calling
application is always in full control of both which syntaxes it supports
and which functions it places in the function table, and so an application
using some other syntax can simply avoid using namespaced functions until
that syntax is updated to understand the new convention.
This initial commit only includes the basic functionality and does not yet
update the specification or specification test suite. It also has only
minimal unit tests of the parser and evaluator. Before finalizing this
in a release we would need to complete that work to make sure everything
is consistent and that we have sufficient regression tests for this new
capability.
|
|
Correct mark handling for some conditional values.
Find correct refinement for overlapping ranges which could not have been
compared with `GreaterThan`. Also map inclusive flags for numeric
ranges.
Correct handling of DefinitelyNotNull collections.
Return a known null early when both conditional branches are null.
|
|
|
|
When attempting to determine the final length range for a conditional
expression with collections, the length values may still be unknown.
Always use `Range()` to get the lower and upper bounds.
|
|
The interactions between value marks and unknown value refinements can be
a little tricky, so this pair of new tests cover two examples of that
interaction that are currently working and ought to stay that way.
|
|
Signed-off-by: Jakub Martin <kubam@spacelift.io>
|
|
We know that a splat expression can never produce a null result, and also
in many cases we can use length refinements from the source collection to
also refine the destination collection because we know that a splat
expression produces exactly one result for each input element.
This also allows us to be a little more precise in the case where the
splat operator is projecting a non-list/set value into a zero or one
element list and we know the source value isn't null. This refinement is
a bit more marginal since it would be weird to apply the splat operator
to a value already known to be non-null anyway, but the refinement might
come from far away from the splat expression and so could still have
useful downstream effects in some cases.
|
|
When ConditionalExpr has an unknown predicate it can still often infer
some refinement to the range of its result by noticing characteristics
that the two results have in common.
In all cases we can test if either result could be null and return a
definitely-not-null unknown value if not.
For two known numbers we can constrain the range to be between those two
numbers. This is primarily aimed at the common case where the two possible
results are zero and one, which significantly constrains the range.
For two known collections of the same kind we can constrain the length
to be between the two collection lengths.
In these last two cases we can also sometimes collapse the unknown into
a known value if the range gets reduced enough. For example, if choosing
between two collections of the same length we might return a known
collection of that length containing unknown elements, rather than an
unknown collection.
|
|
This new concept allows constraining the range of an unknown value beyond
what can be captured in a type constraint. We'll make more use of this
in subsequent commits.
|
|
* [COMPLIANCE] Add Copyright and License Headers
* add copywrite file and revert headers in testdata
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>
|
|
|
|
|
|
|
|
For a long time now we've had a very simplistic error message for the
case of conditional expression result arms not having the same type, which
only works for situations where the two types have differing "friendly
names" down in the cty layer.
Unfortunately due to the typical complexity of the structural type kinds
(object and tuple types) their friendly names are just "object" and
"tuple", which tends to lead us to seemingly-incorrect error messages
like:
The true and false result expressions must have consistent types.
The given expressions are object and object, respectively.
This then is an attempt to use some more specialized messaging in some
of the situations that led to that sort of weird message before. In
particular, this handles:
- both types are object types but their attributes don't match
- both types are tuple types but their elements don't match
- both types are the same kind of collection of either object or tuple
types which don't match
These are the three _shallow_ cases that the previous logic wasn't able to
properly describe. This still leaves unaddressed a hopefully-less-common
case of nested collections with differing structural types in their
depths, but still avoids generating a confusing error message by instead
generating a _very vague but still correct_ error message:
At least one deeply-nested attribute or element is not compatible
across both the 'true' and the 'false' value.
My intent here is to make HCL return something precise enough _most of the
time_, without letting perfect be the enemy of the good. This will
generate some quite obnoxious long messages for particularly complex
nested structures, but so far it appears that such values are relatively
rare inside conditional expressions and so we'll wait to see what arises
in practice before trying to handle those situations more concisely.
Ideally I would like to include some actionable feedback that in some
cases it can help to explicitly convert ambiguously-typed expressions
like "null" or tuples intended to be lists to the intended type, so that
the type unification step has more information to infer the author intent.
However, HCL itself doesn't have any builtins for such conversions and so
today any messaging about that would need to be generated up at the
application layer so the application can refer to whatever functions/etc
it provides for type conversion. It isn't clear how to do that with the
current design, so we'll leave that to be addressed another day.
|
|
When upgrading an unknown splat value, the resulting collection may have
0 elements if the value ends up being `null`. This means the type must
be dynamic.
|
|
When a splat expression is used to upgrade an unknown value to a list
with a single value, the result must be entirely unknown. For example,
if `ukstr` is an unknown string value, then `unkstr.*` could either
result in a list of 1 string value, or an empty list, depending on
whether the result is a string or null.
|
|
When a function returns a functions.ArgError it's supposed to set the
argument index to a valid index within the range of the given argument
slice.
However, we typically try to be robust in the face of
incorrectly-implemented functions, so rather than panicking in that case
as we would've before we'll now return a slightly-less-precise error
message.
The "if i > len(e.Args)-1" case in the previous code here was actually
trying to handle this situation before, but it didn't quite manage to do
it because it was incorrectly referring to e.Args rather than "args"
(so it couldn't take into account expanded arguments) and because it
incorrectly assumed that there would always be more declared parameters
than arguments, which is actually never true in any valid call.
Now we'll handle out of range a little differently for variadic vs.
non-variadic functions. For variadics we'll assume that the function was
trying to talk about the variadic parameter and name that in the error
message. For non-variadics we can't do anything special so we end up
just treating it the same as any other error type, simply stating that
the function call failed without naming a particular argument.
Functions that end up in this error-handling codepath should still be
fixed, because they'll likely end up returning a slightly confusing error
message which doesn't accurately reflect the source of the problem. The
goal here is just avoid a panic in that case.
|
|
Mark objects with keys that are sensitive
|
|
hashicorp/alisdair/marked-for-expression-conditional
hclsyntax: Fix for expression marked conditional
|
|
This adjusts prior behavior that would error to now allow keys that
are marked; those marks are removed in evaluation in order to give
a valid string for the map key, but the resulting object or tuple
is marked as a whole with whatever mark the key contained, ensuring
marks are maintained.
|
|
Using marked values in a for expression conditional expression would
previously panic due to calling `.False()` on the result of the
expression.
This commit makes two changes:
- Unmark the conditional expression value before testing if it is false;
- Merge any marks from the conditional into the resulting marks for the
for expression.
|
|
|
|
|
|
Return an error for invalid for expressions with marks
|
|
When evaluating an HCL expression attempting
to use a marked value as an object key,
return an error rather than falling through
to the cty panic. The error style mimics similar
errors in the area.
|
|
|
|
Functions which accept multiple parameters can be called with the
expansion operator, `...`. When doing so, we must unmark the expanded
argument value before transforming it into a collection of function
arguments. To ensure that any marks applied to the collection are
preserved, we transfer the collection marks to the individual elements
as we build the argument list.
|
|
In conditional expressions that involve a marked value,
Unmark the value before inspecting its truthiness,
to avoid a mark panic in cty
|
|
The rules for the splat operator call for it to return an empty tuple
when its operand is null, but this rule was previously being
overridden by another rule that a value whose type is unknown
causes the operator to return an unknown value of unknown
type.
This was initially reported and discussed in Terraform, under
hashicorp/terraform#26746.
|
|
A for expression over a marked collection should result in a new
collection with the same marks. Previously this would fail with a type
error.
|
|
vals
|
|
Previously functions such as concat() would result in a panic if there
was a null element and a sequence, as in the included test. This PR adds
a check if the error index is outside of the range of arguments and
crafts an error that references the entire function instead of the null
argument.
|
|
Our error message for the ambiguous situation recommends doing this, but
the parser didn't actually previously allow it. Now we'll accept the form
that the error message recommends.
As before, we also accept a template with an interpolation sequence as
a disambiguation, but the error message doesn't mention that because it's
no longer idiomatic to use an inline string template containing just a
single interpolation sequence.
|
|
The main HCL package is more visible this way, and so it's easier than
having to pick it out from dozens of other package directories.
|
