Question about using gdUnit as an API #96

Xananax · 2021-06-16T16:05:36Z

Xananax
Jun 16, 2021

We're building a live editor for Godot which runs in the browser and validates custom user code

We have the editor part working (tweet of the early proto), with error reporting and so on, and we're now deciding how to implement validation.

Instead of reinventing the wheel, I'm looking at gdUnit and I think I like what I see, but I am not clear if it can fit the bill.

I'd have two questions:

Can gdUnit tests run in the browser (in release mode) and be exported with the project?
Is there a small example about how we could use gdUnit from our own custom code, as an API, rather than a full harness? An example case: when a user presses a button, I would like to run a series of tests on a scene's sprites, collect the results, and display them to the user. Again, to be clear, the user in this case is a student, and this is happening in an exported project

Answered by MikeSchulze

Jul 5, 2021

Ok, this means in GdUnit terms that you will have a test suite that will test a scene. A test case will test if a player jumps when a custom button is pressed.

It is still unclear to me when the tests should be executed. According to your example the tests run independently from the actual scene run or?
So the input is not tested in real time?
Which Godot version do you use for the export?

The code to execute a test suite is very simple.

class_name ExecutorExample
extends Node

var _executor :GdUnitExecutor
var _test_suites :Array = Array()

func _ready():
	_executor = GdUnitExecutor.new()
	add_child(_executor)
	# needs to disable the default yielding, it is only need when run in client/s…

View full answer

MikeSchulze · 2021-06-16T17:46:23Z

MikeSchulze
Jun 16, 2021
Maintainer

Hello, thank you for the interest and the look into my project

Can gdUnit tests run in the browser (in release mode) and be exported with the project?

Generally it should be possible to run inside a browser, the code is 99% pure GdScript (running in a browser I never tested).
Running the editor in a browser is an upcoming feature in Godot 4.x and GdUnit3 only runs on Godot 3.x and needs to be switched to the new Godot4 API beforehand.
Currently the release mode is run "externally" and has no debug information, e.g. error lines.
But I suspect that in your use case you need the debug mode inside the editor to get the full report including error lines.

Is there a small example about how we could use gdUnit from our own custom code, as an API, rather than a full harness? An example case: when a user presses a button, I would like to run a series of tests on a scene's sprites, collect the results, and display them to the user. Again, to be clear, the user in this case is a student, and this is happening in an exported project

GdUnit was not designed as an API for third party systems but it should in principle be possible via a custom executor to run dynamic generated tests.
The best example would be GdUnitCmdTool which is a runner with an executor that executes test suites controlled by arguments and collects the reports and saves them as HTML.
If I understand you correctly, you want to run tests from a predefined scene?
So you have the test cases already in your head and you could use predefined
TestSuites depending on the use case.

If you have any further questions, please contact me.

0 replies

MikeSchulze · 2021-07-03T08:15:32Z

MikeSchulze
Jul 3, 2021
Maintainer

Hello @Xananax any questions?
What kind of API do you need, can you give me an example?

1 reply

Xananax Jul 5, 2021
Author

We're building something between Kaboon.JS' guide and ExecuteProgram

Here's an example case I am currently running;

building a course that teaches how to create a platformer is written by an educator. The educator creates a Godot project with missing code, as well as some tests that can verify the platformer behaves as intended (for example, that the Player jumps)
the Godot app, with missing code and tests, gets exported as a web app
later, when a student presses the "run" button in the exported application
a series of tests on a scene's sprites are ran, results are collected
results are displayed to the student, informing them of success or failure

Currently, my custom made test looks like this:

func validate(scene: Node, _script_text: String):
	if not verify(InputMap.has_action("jump"), "no jump action"):
		return
	var player = scene.get_node("Player")
	var position = player.position.y
	Input.action_press("jump")
	yield(get_tree(),"idle_frame")
	yield(get_tree(),"idle_frame")
	if not verify(player.position.y < position, "player's y didn't increase when pressing jump"):
		return
	emit_signal("validation_completed",[])

func verify(statement, message: String) -> bool:
	if statement:
		return true
	emit_signal("validation_completed", [message])
	return false

MikeSchulze · 2021-07-05T15:11:24Z

MikeSchulze
Jul 5, 2021
Maintainer

Ok, this means in GdUnit terms that you will have a test suite that will test a scene. A test case will test if a player jumps when a custom button is pressed.

It is still unclear to me when the tests should be executed. According to your example the tests run independently from the actual scene run or?
So the input is not tested in real time?
Which Godot version do you use for the export?

The code to execute a test suite is very simple.

class_name ExecutorExample
extends Node

var _executor :GdUnitExecutor
var _test_suites :Array = Array()

func _ready():
	_executor = GdUnitExecutor.new()
	add_child(_executor)
	# needs to disable the default yielding, it is only need when run in client/server context 
	_executor.disable_default_yield()
	# optional you can set to stop on first test failure to be fail fast
	_executor.fail_fast(true)
	# register here for executor events to catch reports
	_executor.connect("send_event", self, "_on_executor_event")

# execute a testsuite by path 
# e.g. "res://addons/gdUnit3/test/asserts/GdUnitArrayAssertImplTest.gd"
func execute(path :String) -> void:
	_test_suites.append(GdUnitTestResourceLoader.load_test_suite(path))

func _on_executor_event(event :GdUnitEvent):
	match event.type():
		GdUnitEvent.TESTSUITE_AFTER:
			prints(event.suite_name(), "%dms" % event.elapsed_time())
		
		GdUnitEvent.TESTCASE_AFTER:
			prints(event.test_name(), "failed" if event.is_failed() else "success", "%dms" % event.elapsed_time())
			for element in event.reports():
				var report :GdUnitReport = element
				match report.type():
					GdUnitReport.SUCCESS:
						prints("success", report.message())
					GdUnitReport.FAILURE:
						prints("failed", report.message())

func _process(_delta):
	if _test_suites.empty():
		return
	var test_suite :GdUnitTestSuite = _test_suites.pop_front()
	set_process(false)
	var fs = _executor.execute(test_suite)
	if fs is GDScriptFunctionState:
		yield(fs, "completed")
	set_process(true)

func _on_ToolButton_pressed():
	execute("res://addons/gdUnit3/test/asserts/GdUnitArrayAssertImplTest.gd")

1 reply

Xananax Jul 6, 2021
Author

Ah I see. Thanks, that is interesting!

It is still unclear to me when the tests should be executed. According to your example the tests run independently from the actual scene run or?

The scene is running at the time the user makes a change in the code. Once the user presses "run", this submitted code is checked for syntax using Godot's LSP. If the code passes the LSP check, relevant scripts are changed on the fly in the running scene. Then the scene is submit to the testing suite for arbitrary functional tests prepared by the teacher. The test receives a reference to the running scene, as well as the code as a string (for regex checks or others).

So the input is not tested in real time?

The input is tested in real time, the scene runs, inputs are triggered on it through the tests, and nodes are checked for having moved as we expect them to. The player will actually jump during the test, in the scene the student sees.

I'm open to alternatives though, none of this is set in stone.

Which Godot version do you use for the export?

At the moment we use 3.3.2, but we aren't married to a version. We'll probably update all minors, and almost certainly switch to 4 when it's in beta. Since this is a learning tool, it has to follow the latest Godot version.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Question about using gdUnit as an API #96

{{title}}

Replies: 3 comments 2 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Question about using gdUnit as an API #96

Xananax Jun 16, 2021

Replies: 3 comments · 2 replies

MikeSchulze Jun 16, 2021 Maintainer

MikeSchulze Jul 3, 2021 Maintainer

Xananax Jul 5, 2021 Author

MikeSchulze Jul 5, 2021 Maintainer

Xananax Jul 6, 2021 Author

Xananax
Jun 16, 2021

Replies: 3 comments 2 replies

MikeSchulze
Jun 16, 2021
Maintainer

MikeSchulze
Jul 3, 2021
Maintainer

Xananax Jul 5, 2021
Author

MikeSchulze
Jul 5, 2021
Maintainer

Xananax Jul 6, 2021
Author