Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure strings with only underscores are not processed as Integer #687

Merged
merged 1 commit into from
Sep 11, 2024
Merged

Ensure strings with only underscores are not processed as Integer #687

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions lib/psych/scalar_scanner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,18 @@ class ScalarScanner
# Base 60, [-+]inf and NaN are handled separately
FLOAT = /^(?:[-+]?([0-9][0-9_,]*)?\.[0-9]*([eE][-+][0-9]+)?(?# base 10))$/x

# Taken from http://yaml.org/type/int.html
INTEGER_STRICT = /^(?:[-+]?0b[0-1_]+ (?# base 2)
|[-+]?0[0-7_]+ (?# base 8)
|[-+]?(0|[1-9][0-9_]*) (?# base 10)
|[-+]?0x[0-9a-fA-F_]+ (?# base 16))$/x
# Taken from http://yaml.org/type/int.html and modified to ensure at least one numerical symbol exists
INTEGER_STRICT = /^(?:[-+]?0b[_]*[0-1][0-1_]* (?# base 2)
|[-+]?0[_]*[0-7][0-7_]* (?# base 8)
|[-+]?(0|[1-9][0-9_]*) (?# base 10)
|[-+]?0x[_]*[0-9a-fA-F][0-9a-fA-F_]* (?# base 16))$/x

# Same as above, but allows commas.
# Not to YML spec, but kept for backwards compatibility
INTEGER_LEGACY = /^(?:[-+]?0b[0-1_,]+ (?# base 2)
|[-+]?0[0-7_,]+ (?# base 8)
INTEGER_LEGACY = /^(?:[-+]?0b[_,]*[0-1][0-1_,]* (?# base 2)
|[-+]?0[_,]*[0-7][0-7_,]* (?# base 8)
|[-+]?(?:0|[1-9](?:[0-9]|,[0-9]|_[0-9])*) (?# base 10)
|[-+]?0x[0-9a-fA-F_,]+ (?# base 16))$/x
|[-+]?0x[_,]*[0-9a-fA-F][0-9a-fA-F_,]* (?# base 16))$/x

attr_reader :class_loader

Expand Down
13 changes: 13 additions & 0 deletions test/psych/test_scalar_scanner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,19 @@ def test_scan_strings_ending_with_underscores
assert_equal '100_', ss.tokenize('100_')
end

def test_scan_strings_with_legacy_int_delimiters
assert_equal '0x_,_', ss.tokenize('0x_,_')
assert_equal '+0__,,', ss.tokenize('+0__,,')
assert_equal '-0b,_,', ss.tokenize('-0b,_,')
end

def test_scan_strings_with_strict_int_delimiters
scanner = Psych::ScalarScanner.new ClassLoader.new, strict_integer: true
assert_equal '0x___', scanner.tokenize('0x___')
assert_equal '+0____', scanner.tokenize('+0____')
assert_equal '-0b___', scanner.tokenize('-0b___')
end

def test_scan_int_commas_and_underscores
# NB: This test is to ensure backward compatibility with prior Psych versions,
# not to test against any actual YAML specification.
Expand Down
Loading