Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StackOverflow error on index files #2155

Open
theveriton opened this issue Feb 20, 2024 · 11 comments
Open

StackOverflow error on index files #2155

theveriton opened this issue Feb 20, 2024 · 11 comments

Comments

@theveriton
Copy link

theveriton commented Feb 20, 2024 •
edited
Loading

Suddenly, Perform CodeQL Analysis step has started failing on index files step.

github/codeql-action/analyze@v2

I've tried using v3 too with no luck

Also, I've tried increasing thread heap size from 2M all the way to 512M with no luck

E:\actions-runner-ghec\_work\_tool\CodeQL\2.16.2\x64\codeql\codeql.exe database finalize --finalize-dataset --threads=12 --ram=30002 E:\actions-runner-ghec\_work\_temp\codeql_databases\java
  Picked up JAVA_TOOL_OPTIONS: -Xss512m
  Running pre-finalize script E:\actions-runner-ghec\_work\_tool\CodeQL\2.16.2\x64\codeql\java\tools\pre-finalize.cmd in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver.
  [2024-02-20 03:25:52] [build-stderr] Picked up JAVA_TOOL_OPTIONS: -Xss512m
  [2024-02-20 03:25:52] [build-stderr] Scanning for files in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver...
  [2024-02-20 03:25:52] [build-stderr] E:\actions-runner-ghec\_work\_temp\codeql_databases\java: Indexing files in in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver...
  [2024-02-20 03:25:53] [build-stderr] Running command in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver: [E:\actions-runner-ghec\_work\_tool\CodeQL\2.16.2\x64\codeql\xml\tools\index-files.cmd, E:\actions-runner-ghec\_work\_temp\codeql_databases\java\working\files-to-index18386133814325443289.list]
  [2024-02-20 03:25:53] [build-stdout] [2024-02-20 03:25:53] [build-stderr] Picked up JAVA_TOOL_OPTIONS: -Xss512m
  [2024-02-20 03:25:55] [build-stderr] Picked up JAVA_TOOL_OPTIONS: -Xss512m
  [2024-02-20 03:25:55] [build-stderr] Scanning for files in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver...
  [2024-02-20 03:25:56] [build-stderr] E:\actions-runner-ghec\_work\_temp\codeql_databases\java: Indexing files in in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver...
  [2024-02-20 03:25:56] [build-stderr] Running command in E:\actions-runner-ghec\_work\test-test-appserver\test-test-appserver: [E:\actions-runner-ghec\_work\_tool\CodeQL\2.16.2\x64\codeql\properties\tools\index-files.cmd, E:\actions-runner-ghec\_work\_temp\codeql_databases\java\working\files-to-index182820796413[80](https://github.com/testsomeorg/test-test-appserver/actions/runs/7971172665/job/21760290942#step:6:81)0805[88](https://github.com/testsomeorg/test-test-appserver/actions/runs/7971172665/job/21760290942#step:6:89).list]
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] Picked up JAVA_TOOL_OPTIONS: -Xss512m
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] Exception in thread "main" java.lang.StackOverflowError
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$GroupHead.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$Loop.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$GroupTail.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$BranchConn.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$CharProperty.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$Branch.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$GroupHead.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$Loop.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$GroupTail.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$BranchConn.match(Unknown Source)
  [2024-02-20 03:25:56] [build-stdout] [2024-02-20 03:25:56] [build-stderr] 	at java.base/java.util.regex.Pattern$CharProperty.match(Unknown Source)
@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

Thanks for reporting. Could you include the entire stack trace or at least the bottom part before it gets to the repeating at java.base/java.util.regex.Pattern lines? Hopefully that can tell us where the problem comes from.

@theveriton
Copy link
Author

This is the bottom:

 [2024-02-19 02:40:22] [build-stdout] [2024-02-19 02:40:22] [build-stderr] 	at java.base/java.util.regex.Pattern$Branch.match(Unknown Source)
  [2024-02-19 02:40:22] [build-stdout] [2024-02-19 02:40:22] [build-stderr] 	at java.base/java.util.regex.Pattern$GroupHead.match(Unknown Source)
  [2024-02-19 02:40:22] [build-stdout] [2024-02-19 02:40:22] [build-stderr] 	at java.base/java.util.regex.Pattern$Loop.match(Unknown Source)
  Error: 2-19 02:40:22] [build-stdout] [2024-02-19 02:40:22] [ERROR] Spawned process exited abnormally (code 1; tried to run: [E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\properties\tools\index-files.cmd, E:\actions-runner-ghec-2\_work\_temp\codeql_databases\java\working\files-to-index9666766442730007879.list])
  [2024-02-19 02:40:22] [build-stderr] A fatal error occurred: Exit status 1 from command: [E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\properties\tools\index-files.cmd, E:\actions-runner-ghec-2\_work\_temp\codeql_databases\java\working\files-to-index9666766442730007879.list]
  Error: 2-19 02:40:22] [ERROR] Spawned process exited abnormally (code 2; tried to run: [E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\java\tools\pre-finalize.cmd])
  A fatal error occurred: Exit status 2 from command: [E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\java\tools\pre-finalize.cmd]
  Error: Encountered a fatal error while running "E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\codeql.exe database finalize --finalize-dataset --threads=6 --ram=14437 E:\actions-runner-ghec-2\_work\_temp\codeql_databases\java". Exit code was 2 and error was: A fatal error occurred: Exit status 2 from command: [E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\tools\win64\runner.exe, cmd.exe, /C, type, NUL, &&, E:\actions-runner-ghec-2\_work\_tool\CodeQL\2.16.2\x64\codeql\java\tools\pre-finalize.cmd]. See the logs for more details.

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

I was afraid of that. I suppose a trace for a stack overflow is too long to show the non-repeating start of it. What project are you running on? Is it open source? I just noticed on a local test that I get a similar stacktrace when running CodeQL on https://github.com/apache/hive .

@aibaars aibaars mentioned this issue Feb 20, 2024
Closed
@theveriton
Copy link
Author

No, it isn't open source, i've redacted the repo and org info in the logs. It's a self hosted runner, will the logs be anywhere on that?

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

@theveriton Does this problem occur all the time or just this once? I spotted a case like this when analysing apache/hive, but it didn't happen in the dozen re-runs I made. This makes it hard to find the cause.

The easiest way to grab the logs is the re-run a workflow with debug logging enabled. That causes the codeql-action to upload a debug artifact. You could also try to locate the "database" folders in the "RUNNER_TEMP" folder on the self-hosted runner. However, these are cleared up by subsequent jobs.

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

@theveriton Those links do not work (luckily). The debug artifacts contain logs, source code and other potentially sensitive data. Since this is a closed source repository you probably do not want to share that on a public issue. You can download and unzip the artifact to check the various logs folders for potentially interesting error messages. Alternatively, you can open an Enterprise support ticket and share the artifact privately with GitHub's support engineers.

Another thing to try would be to set the tools: property of the codeql-action/init step to the URL of last month's version of CodeQL. If we can figure out the last working/first failing versions then that would help narrow down the search for the cause of the problem.

@theveriton
Copy link
Author

Awesome! It started working with v2.15.5, doesn't work for 2.16.0 and beyond.

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

Thanks, glad things are working for you now. Hopefully the team can figure out what happened between 2.15.5 and 2.16.0 .

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

It looks like since 2.16.0 CodeQL extracts all *.properties files by default. Do you have a malformed properties file in your repository by any chance? Could you try adding:

env:
  LGTM_INDEX_PROPERTIES_FILES: "false"

to your workflow file and go back to using the 2.16 release?

@theveriton
Copy link
Author

That also worked! v2.16.2...we do have a bunch of property files.

@aibaars
Copy link
Collaborator

aibaars commented Feb 20, 2024

I think the following regular expression may have performance problems when run on inputs that are not really properties files.

We'll look into improving things. Thanks again for reporting this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aibaars @theveriton