Implement testing for server package by hpoeche · Pull Request #627 · eclipse-basyx/basyx-python-sdk

hpoeche · 2026-09-07T09:34:27Z

Changes

This PR adds unit and integration test to the previously untested server package. In detail the following test setup is used

Unit Testing

Unit tests are primarily written against the served API, employing werkzeug.test.Client to test the werkzeug based API interfaces.

All three server interfaces (repository, discovery, registry) are tested this way.

Format specific testing

All implemented endpoints are tested with all supported Content-Types and for all possible responses.
Responses are not deserialized using the SDK adapter, because round-trip serialization-deserialization is already tested in the SDK. However, the correctness of responses are still verified by parsing them using standard libraries (jsonand lxml.etree) and checking for specific expected data.
To reduce repetition of test for both Content-Types application/json and application/xml, test/interfaces/format_utils.py defines a format-agnostic wrapper FormatClient of werkzeug.test.Client that has subclasses JsonFormatClient and XmlFormatClient handling serialization of request bodies as well as parsing and checking as described above.
The test cases are written against this format-agnostic FormatClient. Function decorators with_json_client/with_xml_client are used to create separate test cases with the correct client at runtime.

Coverage

In general, endpoints are tested for successful response and all error codes, defined in the specification. More specifically:

GET endpoints are tested for
- 200: Check if only requested elements are returned
- 404: Fail for non-existing elements
- If pagination is supported: Check repeated paginated requests produce complete result set
POST endpoints are tested for
- 201: Check, if object is stored to the used *ObjectStore
- 400: Check, if malformed objects are rejected
- 409: Check, if already existing object are rejected
PUT endpoints are tested for
- 204: Check, if object in the *ObjectStore is altered correctly
- 404: Fail for non-existing element
DELETE endpoints are tested for
- 204: Check correct deletion in the *ObjectStore
- 404: Fail for non-existing elements

Additional Tests

While the test above test if the endpoints behavior is compliant to the specification, some additional tests directly test the code in base.py:

The pagination logic is centrally defined in base.py, therefore we extensively test it separate for correct logic (pages build a partition of complete set) and parameter handling.
As the test above do not deserialize the result, the correct serialization of JsonResponse and XmlResponse is tested separately
Server side deserialization (HTTPApiDecoder), is already tested in the test cases above.

Entrypoint Testing

The entrypoints run_registry.py and run_discovery.py (from server/app/services) that start up the respective server profile, configure the storage and the API base path via environment variables. We test that setting these variables results in the correct storage setup inside the tests in server/test/services. We need to use importlib to run the code because the entrypoints are written as scripts, that run their code at import time. Therefore no coverage data can be collected.

Integration Testing

To ensure correct setup of uWSGI, nginx and supervisor inside the docker container, we run basic integration tests.

Therefore, server/test/docker_integration/ has one test module per profile (repository, registry, discovery) that runs against a real, already-running server, checking /description and a full create/retrieve/delete roundtrip per profile.

By default these tests skip if no server is reachable (like test_couchdb.py in the SDK). Setting REQUIRE_SERVER_INTEGRATION_TESTS makes them fail instead of skip. We use this setting in the server-docker CI job, which now builds and tests all three profiles.

This is a draft to test unittest implementations for the server. Implement unittests using `werkzeug.test.Client` to test the `/shells` endpoint, as example. Each endpoint and method is tested for success and possible failures. The object store is reset prior to every test case. Tests are repeated for `application/json` and `application/xml` Content-Types. Therfore test are written against an abstract `FromatClient` that covers the details of (de-)serialization behind a simple API for requesting and parsing. Therefore the base class defining the test cases (`_ShellsEndpointTest`) is disabled for testing. Two subclasses are derived from this class, one for each format, that define the correct `FormatClient` and execute the tests.

…umbnail endpoint.

…skip pattern

…ation thumbnail endpoint." This reverts commit 496190f, which held the content of eclipse-basyx#618. This was added for testing purpose only. Merge `develop` into this branch, after the PR was closed to obtain the same result.

For now the `/submodel-elements` paths are excluded

Added test class `TestPagination` to `test_base.py` that ensures pagination by following `cursor` value correctly assembles all items. Additionally, all endpoints, that should support pagination are checked if they do so.

…g in CI

… test

To separate testing of the pagination logic from working endpoints, the shared function for creatin paginated responses is now tested directly. The base tests on paginated endpoints remain.

…rites

In the first version of the tests for the Discovery API, when data needed to be added to the DiscoveryStore, this was done through the `POST /lookup/shells/<aasId>` endpoint. To decouple endpoint tests from each other, the data insertion is now done directly via the DiscoveryStore.

…clude testing

hpoeche · 2026-10-05T10:00:40Z

Before we merge, we must check if #642 is already resolved and adapt our changes if so.

s-heppner

Can someone please also fix the merge conflicts?

s-heppner · 2026-10-05T09:34:03Z

+    def assert_error(self, response: TestResponse, status_code: int) -> None:
+        self.assertEqual(status_code, response.status_code, msg=response.get_data(as_text=True))
+        self.assertIn("success", response.get_data(as_text=True), msg=response.get_data(as_text=True))
+


What if there's a string like "This action was unsuccessful." somewhere in the response body? I don't doubt that this isn't the case rigt now, but this check seems weak and might falsely state success when there was none.

Thank you for checking. assert_error() now parses the actual success field instead of doing a substring check on the text.

s-heppner · 2026-10-05T09:35:20Z

+# to that file to override the defaults defined here.
+
+[server]
+url = http://localhost:8080/api/v3.1


Isn't this a problem to hardcode this, value, when it should use the single point of truth from versions.toml?

Well spotted, I fixed this in 960c0fb.

Drop the per-descriptor `DESCRIPTOR_TYPE_TO_STRING` lookup and nested helper function; serialize both descriptor lists once and inject the literal modelType string in two plain loops instead, matching the pattern already used in `services/support.py`.

Resolved conflicts in CONTRIBUTING.md

Splits the definition of the server `url` value into `host` and `base_path` in the `test_config.default.ini`. The values are concatenated in `test_helpers.py` to define the `url` value. If `base_path` is set to `DEFAULT`, its value is read from `versions.toml`. This is the default value for `base_path` so the tests do not fail on API version bumps.

hpoeche and others added 30 commits August 28, 2026 18:30

Implement and test GET/PUT/DELETE for the shells asset-information th…

496190f

…umbnail endpoint.

extract abstract RepositoryEndpointTestBase

d21a139

use decorators to indicate which clients to use for which test case

e51bd38

apply decorators everywhere

19759db

Structure tests into multiple files

35fca6d

Add live server integration test, mirroring test_couchdb.py's config/…

6d7d096

…skip pattern

Include shells query parameter tests into

67d8095

fix relative import

74acaf6

Add tests for /submodels

4c08d30

For now the `/submodel-elements` paths are excluded

Add tests for /submodel/.../submodel-elements

edebc04

Fix CI port and run Docker integration tests

bb2346e

add missed (error) paths

b5b684d

Add unittests for /concept-description endpoints

fcefaf3

Test pagination of results

cb26b0a

Added test class `TestPagination` to `test_base.py` that ensures pagination by following `cursor` value correctly assembles all items. Additionally, all endpoints, that should support pagination are checked if they do so.

fix mypy and ruff errors

1326d6a

Make Docker integration tests fail loudly instead of silently skippin…

22ff327

…g in CI

Add duplicate-POST, update, and not-found cases to Docker integration…

134c33a

… test

Implement tests for registry

d2ca6c4

Implement tests for discovery

ddccd15

Change pagination test to directly call function

b91c59a

To separate testing of the pagination logic from working endpoints, the shared function for creatin paginated responses is now tested directly. The base tests on paginated endpoints remain.

Fix DictDescriptorStore missing commit() causing 500s on descriptor w…

46161d4

…rites

Add Docker integration tests for the registry and discovery profile

54b5a6e

Concrete testing of APIResponse serialization

9086566

Fix CI server docker job

5208257

Clearify comment

82e4fd4

Move integration tests to own module

cf66bb5

Drop integration tests duplicated by endpoint test

29fdb73

paul-gerber-svg and others added 8 commits September 19, 2026 15:59

Address review feedback on server test suite

dedfc4d

Add missing copyright headers

34a420b

Test run_repository.py loads files according to env variables

aeaa506

Test run_registry.py loads and overwrites files correctly

4105cce

Test loads file on run and saves on exit

e138bd9

Fix: Rename test class for run_registry.py

7a791f3

Update the CONTRIBUTING.md to run server tests

c807188

Resolve line length violations

25d6bfd

JAB1305 reviewed Sep 25, 2026

View reviewed changes

paul-gerber-svg and others added 5 commits September 27, 2026 12:08

Address review feedback on the server test suite

fb8181e

Drop module docstring for consistency

7813a07

Correct comments around server integration tests

0fede30

Use relative imports for test modules everywhere

1198fe9

Mark _InMemoryDescriptorStore as workaround for eclipse-basyx#626

149a6dc

hpoeche mentioned this pull request Sep 27, 2026

'DictDescriptorStore' add no-op commit #642

Merged

hpoeche marked this pull request as ready for review September 27, 2026 17:20

hpoeche and others added 2 commits October 2, 2026 10:02

Draft for testing README.md

fe75bec

Add Integration Test to testing README and change server README to in…

ab35164

…clude testing

s-heppner requested changes Oct 5, 2026

View reviewed changes

Fix false-positive on substring match

daa239c

This was referenced Oct 6, 2026

server: Implement the /serialization route #622

Open

Implement GET/PUT/DELETE for /shells/asset-information/thumbnail #618

Open

Adjust discovery status codes to align with specification #641

Open

paul-gerber-svg and others added 5 commits October 6, 2026 16:13

Add section about unit tests to README.md

3fd08af

Merge remote-tracking branch 'upstream/develop' into feat/server-tests

f9dec86

Resolved conflicts in CONTRIBUTING.md

Remove workaround for DictDescriptorStore (eclipse-basyx#626)

2a8eb69

hpoeche requested a review from s-heppner October 7, 2026 10:06

Navigation Menu

Repository navigation

Implement testing for server package - #627

Implement testing for server package#627
hpoeche wants to merge 62 commits into
eclipse-basyx:developfrom
rwth-iat:feat/server-tests

hpoeche commented Sep 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hpoeche commented Oct 5, 2026

Uh oh!

s-heppner left a comment

Uh oh!

s-heppner Oct 5, 2026

Uh oh!

paul-gerber-svg Oct 6, 2026

Uh oh!

s-heppner Oct 5, 2026

Uh oh!

hpoeche Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

hpoeche commented Sep 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Unit Testing

Format specific testing

Coverage

Additional Tests

Entrypoint Testing

Integration Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hpoeche commented Oct 5, 2026

Uh oh!

s-heppner left a comment

Choose a reason for hiding this comment

Uh oh!

s-heppner Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

paul-gerber-svg Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

s-heppner Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

hpoeche Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hpoeche commented Sep 7, 2026 •

edited

Loading