Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip search and replace on objects that can't deserialize safely #192

Merged
merged 26 commits into from
Dec 19, 2023
Merged

Skip search and replace on objects that can't deserialize safely #192

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
aff9a8f
skip deserialization of S&R objects that return TypeErrors
MarkBerube Nov 27, 2023
4ef93f7
Update SearchReplacer.php based off PHP CS
MarkBerube Nov 27, 2023
fde70ec
Update SearchReplacer.php with changes from PHPCS
MarkBerube Nov 27, 2023
2d7ce67
adding skip on phpcs incompat as testing shows its safe for PHP5.6 & …
MarkBerube Nov 28, 2023
9c46168
fixing small typo in behat scenario
MarkBerube Nov 28, 2023
afa1d7b
removing unncessary comment
MarkBerube Nov 28, 2023
7c373b4
removing unncessary comments
MarkBerube Nov 28, 2023
8f338d9
Update features/search-replace.feature scenario description
MarkBerube Nov 28, 2023
f6e1a3e
skip over replacements if TypeError is executed.
MarkBerube Nov 28, 2023
57cb6eb
converting obj to array to avoid errors with funky/blank objects on d…
MarkBerube Nov 29, 2023
6ff1d6b
adding safe object to search & replace over in test
MarkBerube Nov 29, 2023
d6437f2
removing array cast & testing serialized, safe object to confirm seri…
MarkBerube Nov 30, 2023
ac1ae3b
catch errors in unsafe php objects as they are iterated on
MarkBerube Dec 1, 2023
8889a18
Fix PHPCS issue
schlessera Dec 18, 2023
9404019
Include reason in error messages
schlessera Dec 18, 2023
251042b
Avoid casting objects to string
schlessera Dec 18, 2023
b3a1734
Split into two separate tests per PHP version
schlessera Dec 18, 2023
42ef6c2
Fully form forwarded exception
schlessera Dec 18, 2023
d6430f9
Be precise about PHPCS ignore
schlessera Dec 18, 2023
586522b
Add clarifying comment about type error
schlessera Dec 18, 2023
4509c7d
Add another clarifying comment
schlessera Dec 18, 2023
9c12684
Adapt warning for PHP < 8.0
schlessera Dec 18, 2023
e915dd8
Fix warning text in test
schlessera Dec 18, 2023
813403e
Fix bad anntotation
schlessera Dec 19, 2023
43abfb5
Add safeguard for partial processing to not be counted as a success
schlessera Dec 19, 2023
da1acc7
Skip broken SQLite testing for now
schlessera Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions features/search-replace.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1106,6 +1106,28 @@ Feature: Do global search/replace
a:1:{i:0;O:10:"CornFlakes":0:{}}
"""

Scenario: Warn and ignore type-hinted objects that have some error in deserialization

Given a WP install
And I run `wp db query "INSERT INTO wp_options (option_name,option_value) VALUES ('cereal_isation','O:13:\"mysqli_result\":5:{s:13:\"current_field\";N;s:11:\"field_count\";N;s:7:\"lengths\";N;s:8:\"num_rows\";N;s:4:\"type\";N;}')"`
And I run `wp db query "INSERT INTO wp_options (option_name,option_value) VALUES ('cereal_isation_2','O:8:\"mysqli_result\":5:{s:13:\"current_field\";i:1;s:11:\"field_count\";i:2;s:7:\"lengths\";a:1:{i:0;s:4:\"blah\";}s:8:\"num_rows\";i:1;s:4:\"type\";i:2;}')"`

When I try `wp search-replace mysqli_result stdClass`
Then STDERR should contain:
"""
Warning: Skipping an inconvertible serialized object: "O:13:"mysqli_result":5:{s:13:"current_field";N;s:11:"field_count";N;s:7:"lengths";N;s:8:"num_rows";N;s:4:"type";N;}", replacements might not be complete.
"""
And STDOUT should contain:
"""
Success: Made 1 replacement.
"""

When I run `wp db query "SELECT option_value from wp_options where option_name='cereal_isation_2'"`
Then STDOUT should contain:
"""
O:8:"stdClass":5:{s:13:"current_field";i:1;s:11:"field_count";i:2;s:7:"lengths";a:1:{i:0;s:4:"blah";}s:8:"num_rows";i:1;s:4:"type";i:2;}
"""
MarkBerube marked this conversation as resolved.
Show resolved Hide resolved

Scenario: Regex search/replace with `--regex-limit=1` option
Given a WP install
And I run `wp post create --post_content="I have a pen, I have an apple. Pen, pine-apple, apple-pen."`
Expand Down
26 changes: 19 additions & 7 deletions src/WP_CLI/SearchReplacer.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,24 @@ private function run_recursively( $data, $serialised, $recursion_level = 0, $vis
}
}

// The error suppression operator is not enough in some cases, so we disable
// reporting of notices and warnings as well.
$error_reporting = error_reporting();
error_reporting( $error_reporting & ~E_NOTICE & ~E_WARNING );
$unserialized = is_string( $data ) ? @unserialize( $data ) : false;
error_reporting( $error_reporting );
try {
// The error suppression operator is not enough in some cases, so we disable
// reporting of notices and warnings as well.
$error_reporting = error_reporting();
error_reporting( $error_reporting & ~E_NOTICE & ~E_WARNING );
$unserialized = is_string( $data ) ? @unserialize( $data ) : false;
error_reporting( $error_reporting );

} catch ( \TypeError $e ) { // phpcs:ignore
\WP_CLI::warning(
sprintf(
'Skipping an inconvertible serialized object: "%s", replacements might not be complete.',
$data
)
);

throw new Exception();
}

if ( false !== $unserialized ) {
$data = $this->run_recursively( $unserialized, true, $recursion_level + 1 );
Expand All @@ -107,7 +119,7 @@ private function run_recursively( $data, $serialised, $recursion_level = 0, $vis
)
);
} else {
foreach ( $data as $key => $value ) {
foreach ( (array) $data as $key => $value ) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it will have negative unintended consequences and should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries, removed with the latest commit!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it did lead to these incompatibilities that I don't have a clear fix for yet: https://github.com/wp-cli/search-replace-command/actions/runs/7053576340/job/19201047750 in these specifically failing PHP versions the sql object ends up like this:

object(mysqli_result)#1 (0) {}

Copy link
Contributor Author

@MarkBerube MarkBerube Dec 1, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem here is that foreaching directly over an object is not always safe in PHP and these objects like above show why. Objects in PHP don't always allow iteration and functions like is_iterable() check for the instance of \Traversable on objects & aren't exactly reliable.

So instead of casting, I've found this to be the safest way to iterate through these objects:

try {
foreach ( $data as $key => $value ) {
	$data->$key = $this->run_recursively( $value, false, $recursion_level + 1, $visited_data );
}
} catch ( \Error $e ) {
	\WP_CLI::warning(
	sprintf(
	'Skipping an inconvertible serialized object: "%s", replacements might not be complete.',
	$data
	)
);

throw new Exception();

...

currently works in all versions of PHP I've tested, but I'll need to change my behat tests to account for specific php versions reporting this as a warning instead of a fatal error.

edit: actually just due to being warnings, using the try{} block seems successful in PHP 7.2, 7.4, 8.0 with these changes.

$data->$key = $this->run_recursively( $value, false, $recursion_level + 1, $visited_data );
}
}
Expand Down
Loading