More logs to verify the correct credential is used by target_experiment.py
#13042
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DavidKorczynski
approved these changes
Feb 14, 2025
This reverts commit 50c913e.
12 tasks
DonggeLiu
added a commit
to google/oss-fuzz-gen
that referenced
this pull request
Feb 23, 2025
Fixing some issues revealed by full agent experiments earlier:
1. [x] LLM-generated build scripts do not save fuzz target binary into
the correct path.
2. [x] Use default build script in the code fixing prompt in this
scenario:
1. The default build scripts builds successfully but failed other checks
(i.e., reference), and
2. The LLM-generated build script does not work.
3. [x] Selectively use the default built script and the LLM-generated
built script, depending which is better.
4. [x] Use different code-fixing prompts based on which built script and
which result it is:
* default or LLM built script
* No reference, no binary, or compilation failure
5. [x] Backup human-writtent `/src/build.sh` to `/src/build.bk.sh` in
agent's containers in case LLM wants to reuse it in the new build
script.
* Create the same copy for fuzzing execution.
6. [x] Hide the compile command to prevent LLM from reusing it in the
inspection tool and be distracted by irrelevant errors. E.g.:
* The inspection container always runs compile before LLM analysis.
Rerunning it may fail in some projects due to an existing
/src/<project>/build directory.
7. [x] Prompt use example fuzz target in the language the same as the
generated fuzz target, (not the project).
* Also dynamically adjust instructions in priming. Do not leave LLM to
judge which language the fuzz target is.
8. [x] Remove the agent log when receiving fuzz targets.
9. [x] Do not restrict LLM to send one bash command per query.
Also need to:
1. [ ] Use SemanticAnalyzer in agent workflow, at least to ensure the
last Result is Analysis Result.
2. [ ] Add an Enhancer in agent workflow.
3. [ ] Use service account in GKE, hopefully this will solve the
[`Service Unavailable`
problem](google/oss-fuzz#13042).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
OSS-Fuzz-Gen experiment cloud build does not use service account.
Meanwhile, its cloud builds fails occasionally due to
Service Unavailable:This PR prints information about
GOOGLE_APPLICATION_CREDENTIALSand the credentials used by the cloud build to help debug why service account is not used and if using service account can resolve the 'service unavailable' error.