Open Zaak Documentation

Open Zaak is a modern, open-source data- and services-layer to enable zaakgericht werken, a Dutch alternative to case management. Open Zaak offers structured data storage and services that implement the VNG standards for “API’s voor Zaakgericht werken” in line with the Common Ground model.

Getting Started

To get you started, you might find some of these links relevant:

• New to Open Zaak? Have a look at the Introduction

• New to the VNG standards for “API’s voor Zaakgericht werken”? Read up on the API-specifications.

• Want to get started with Open Zaak yourself? See Installation.

• Want to configure or manage data in Open Zaak? Read the Manual.

• Need help with Open Zaak? Contact Support.

• Are you a developer? Head over to Development!

Open Zaak is and only uses Open-source.

Introduction

Open Zaak is a modern, open-source data- and services-layer to enable zaakgericht werken, a Dutch alternative to case management. Open Zaak offers structured data storage and services that implement the VNG standards for “API’s voor Zaakgericht werken” in line with the Common Ground model.

Open Zaak exposes several API’s to store and retrieve data:

• Zaken API (case instances)

• Documenten API (documents)

• Catalogi API (case types)

• Besluiten API (decisions)

• Autorisaties API (authorizations)

The Notificaties API is required for Open Zaak to work but is available as a separate package, Open Notificaties.

Open Zaak is based on the API reference implementations by VNG Realisatie to create a production-grade product that can be used by municipalities.

Architecture

Open Zaak is based on the reference implementation of the “API’s voor Zaakgericht werken” made by VNG Realisatie. The overall architecture remains faithful to the Common Ground principles and all API specifications.

The architecture of Open Zaak focusses on excellent performance, optimal stability and to guarantee data integrity.

To that end, Open Zaak combines the “API’s voor Zaakgericht werken” that are essentially tightly coupled, into a single product. This allows for major performance improvements since related objects (like a BESLUIT for a ZAAK) do not need to fetched over the network but can be directly obtained from the database. This also guarantees data integrity on database level, rather than on service (API) level.

In addition, Open Zaak uses caching wherever possible to prevent needless requests over the netwerk to fetch data from external API’s. Data integrity can not be guaranteed on database level when relations are created to external API’s. In this case, data integrity is enforced on service level as much as possible.

The use of external API’s is fully supported in Open Zaak, even for API’s that are also offered by Open Zaak itself. For example, a ZAAK in Open Zaak, available via the Zaken API can have a DOCUMENT that is accessible via an external Documenten API from another vendor. The only requirement is that all API’s adhere to VNG standards for “API’s voor Zaakgericht werken”.

No permanent copies are made of original sources in Open Zaak as dictated by the Common Ground principles.

Overview

Who’s behind Open Zaak?

The Open Zaak project was initiated shortly before the 1.0 release of the VNG standards for “API’s voor Zaakgericht werken”, on August 1, 2019, by 9 municipalities:

• Amsterdam

• Rotterdam

• Utrecht

• Tilburg

• Arnhem

• Haarlem

• ‘s-Hertogenbosch

• Delft

• Hoorn, Medemblik, Stede Broec, Drechteland, Enkhuizen (SED)

Using Dimpact as a legal entity, they formed a project to develop a modern, open-source data- and services-layer to enable zaakgericht werken, in line with the Common Ground model.

Open-source

Open Zaak is open-source and available under the EUPL license.

In addition, this project makes use of various open-source Python libraries and npm packages under the hood.

Public Code

The Open Zaak project benefits from the expertise of the Foundation for Public Code who helps to scale our codebase, community and development proces. With their codebase stewardship program, we are working on becomming fully compliant.

Open-Zaak and the Standard for Public Code

Code in the Open

• [x] Open Zaak development happens in the open on Github

Requirement	meets	links and notes
All source code for any policy and software in use (unless used for fraud detection) MUST be published and publicly accessible.	yes	code, VNG/GEMMA2 policy linked in README
Contributors MUST NOT upload sensitive information regarding users, their organization or third parties to the repository.	yes	2020-05-12 review by @EricHerman; ISO 27001
Any source code not currently in use (such as new versions, proposals or older versions) SHOULD be published.	yes	releases, docker hub tags
The source code MAY provide the general public with insight into which source code or policy underpins any specific interaction they have with your organization.

Bundle policy and source code

• [x] Policy bundled, tested, and linked in the documentation, some policy not available in English

Requirement	meets	links and notes
A codebase MUST include the policy that the source code is based on.	yes	API specs embeded in the codebase, and linked from documentation
A codebase MUST include all source code that the policy is based on.		"yes" or "not applicable", do we consider it based on the API?
All policy and source code that the codebase is based on MUST be documented, reusable and portable.	yes	code dependencies are OSI
Policy SHOULD be provided in machine readable and unambiguous formats.	yes	OpenAPI is in machine readable yaml format
Continuous integration tests SHOULD validate that the source code and the policy are executed coherently.	yes	Travis CI config, API Test Platform

Create reusable and portable code

• [x] designed to be reusable from the start

Requirement	meets	links and notes
The codebase MUST be developed to be reusable in different contexts.	yes	Designed to be so from the start.
The codebase MUST be independent from any secret, undisclosed, proprietary or non-open licensed code or services for execution and understanding.	yes	installation supports docker, kubernetes, vmware appliances, bare-metal is possible
The codebase SHOULD be in use by multiple parties.	yes	Deployed in multiple sandbox environments (e.g: Utrecht is testing it, others Den Haag, Delft looking at it.)
The roadmap SHOULD be influenced by the needs of multiple parties.	yes	Dimpact, market consultation
Configuration SHOULD be used to make code adapt to context specific needs.
Codebases SHOULD include a publiccode.yml metadata description so that they’re easily discoverable.	yes	publiccode.yml
Code and its documentation SHOULD NOT contain situation-specific information.	yes	Some GCloud examples but nothing required; no credentials and documentation suggests using secret generators

Welcome contributors

• [ ] compliant with this criterion.

Requirement	meets	links and notes
The codebase MUST allow anyone to submit suggestions for changes to the codebase.	yes	requests
The codebase MUST include contribution guidelines explaining how contributors can get involved, for example in a CONTRIBUTING file.	yes	CONTRIBUTING, Documentation
The codebase SHOULD advertise the committed engagement of involved organizations in the development and maintenance.	yes	Readme
The codebase SHOULD document the governance of the codebase, contributions and its community, for example in a GOVERNANCE file.	no	draft GOVERNANCE file
The codebase SHOULD have a publicly available roadmap.	no	Issues are not yet collected into a roadmap view; tech-debt and tech-wishlist not yet collected into a technical roadmap
The codebase MAY include a code of conduct for contributors.	no	no email address in the Code of conduct to report incidents

Make contributing easy

• [x] compliant with this criterion.

Requirement	meets	links and notes
The codebase MUST have a public issue tracker that accepts suggestions from anyone.	yes	issues
The codebase MUST include an email address for security issues and responsible disclosure.	yes	security@maykinmedia.nl
The documentation MUST link to both the public issue tracker and submitted codebase changes, for example in a README file.	yes	Documentation, CONTRIBUTING.md
The project MUST have communication channels for users and developers, for example email lists.	yes	Mailing list, github issues, VNG slack channel (requires an invite)
The documentation SHOULD include instructions for how to report potentially security sensitive issues on a closed channel.	yes	SECURITY.rst

Maintain version control

• [ ] Good, but let’s wait for a commit template

Requirement	meets	links and notes
You MUST have a way to maintain version control for the code.	yes	GitHub
All files in a codebase MUST be version controlled.	yes	git
All decisions MUST be documented in commit messages.		Generally good, some room for improvement, commit template may help
Every commit message MUST link to discussions and issues wherever possible.		yes for non-trivial commits
You SHOULD group relevant changes in commits.	yes	PRs
You SHOULD mark different versions of the codebase, for example using revision tags or textual labels.	yes	releases
You SHOULD prefer file formats that can easily be version controlled.	yes	mostly code and Restructured Text or Markdown

Require review of contributions

• [x] Practices are good, would like to see explicit docs

Requirement	meets	links and notes
All contributions that are accepted or committed to release versions of the codebase MUST be reviewed by another contributor.	yes	PRs
Reviews MUST include source, policy, tests and documentation.	yes	repo is configured, tests span, practices right, yet documentation could be more explicit on this point
Reviewers MUST provide feedback on all decisions made and the implementation in the review.	yes	documentations/contrib guidelines could copy-paste from the employement handbook, PRs
Contributions SHOULD conform to the standards, architecture and decisions set out in the codebase in order to pass review.	yes	PRs, django best practices
Reviews SHOULD include running both the code and the tests of the codebase.	yes	CI and manual validation required, not clearly documented yet (in handbook)
Contributions SHOULD be reviewed by someone in a different context than the contributor.		(some) not applicable yet
Version control systems SHOULD not accept non-reviewed contributions in release versions.	yes	master branch protected, release process
Reviews SHOULD happen within two business days.	yes	no official policy, is the practice
Reviews MAY be performed by multiple reviewers.		mostly

Document your objectives

• [x] Objectives are clear in the introduction documenation

Requirement	meets	links and notes
The codebase MUST contain documentation of its objectives – like a mission and goal statement – that is understandable by designers and developers so that they can use or contribute to the codebase.	yes	"Open Zaak is based on the API reference implementations by VNG Realisatie to create a production-grade product that can be used by municipalities." introduction
The codebase SHOULD contain documentation of its objectives understandable by policy makers and management.	yes
The codebase MAY contain documentation of its objectives for the general public.		could elaborate for general public: Common Ground, data security, GDPR requests and such

Document your code

• [x] documentation is good across the board, “recipies” would be nice

Requirement	meets	links and notes
All of the functionality of the codebase – policy as well as source – MUST be described in language clearly understandable for those that understand the purpose of the code.	yes	additional docs are generated from code comments
The documentation of the codebase MUST contain: a description of how to install and run the source code, examples demonstrating the key functionality.	yes	getting started, post-install checklist, improvement: add demo fixtures/instructions
The documentation of the codebase SHOULD contain: a high level description that is clearly understandable for a wide audience of stakeholders, like the general public and journalists, a section describing how to install and run a standalone version of the source code, including, if necessary, a test dataset, examples for all functionality.		documentation, would like to see standard recipies and examples of more fuctionality
There SHOULD be continuous integration tests for the quality of your documentation.	yes	link checks, build checks, Travis config
The documentation of the codebase MAY contain examples that make users want to immediately start using the codebase.
You MAY use the examples in your documentation to test the code.

Use plain English

• [ ] compliant with this criterion.

Requirement	meets	links and notes
All code and documentation MUST be in English.		manual in Dutch
Any translation MUST be up to date with the English version and vice versa.		TODO: add translations of user-facing texts (NL -> EN) (makemessages)
There SHOULD be no acronyms, abbreviations, puns or legal/domain specific terms in the codebase without an explanation preceding it or a link to an explanation.		Domain specific Dutch terms could be in a glossary which is also translated in English.
The name of the project or codebase SHOULD be descriptive and free from acronyms, abbreviations, puns or branding.	yes
Documentation SHOULD aim for a lower secondary education reading level, as recommended by the Web Content Accessibility Guidelines 2.		Would be good to get an evaluation of this prior to investing in translation.
Any code, documentation and tests MAY have a translation.

Use open standards

• [x] compliant with this criterion.

Requirement	meets	links and notes
For features of a codebase that facilitate the exchange of data the codebase MUST use an open standard that meets the Open Source Initiative Open Standard Requirements.	yes	The Zaakgericht Werken in het Gemeentelijk Gegevenslandschap meets the 5 criteria of the Open Standards Requirement for Software
If no existing open standard is available, effort SHOULD be put into developing one.	n/a
Standards that are machine testable SHOULD be preferred over those that are not.	yes	see test suite
Functionality using features from a non-open standard (one that doesn’t meet the Open Source Initiative Open Standard Requirements) MAY be provided if necessary, but only in addition to compliant features.
All non-compliant standards used MUST be recorded clearly in the documentation.	n/a
The codebase SHOULD contain a list of all the standards used with links to where they are available.	yes

Use continuous integration

• [ ] compliant with this criterion.

Requirement	meets	links and notes
All functionality in the source code MUST have automated tests.		96% codecoverage, small amount of admin UI code not tested
Contributions MUST pass all automated tests before they are admitted into the codebase.	yes	github checks
Contributions MUST be small.		refer to contributing guidelines (single issue, single PR solves single issue)
The codebase MUST have active contributors.	yes	pulse
Source code test and documentation coverage SHOULD be monitored.	yes
Policy and documentation MAY have testing for consistency with the source and vice versa.	yes
Policy and documentation MAY have testing for style and broken links.	yes	docs tests, flake8/isort/black

Publish with an open license

• [x] Remove confusion in the documentation footer

Requirement	meets	links and notes
All code and documentation MUST be licensed such that it may be freely reusable, changeable and redistributable.	yes	copyright marks in the footer? -> check if we can add license to footer (sphinx conf), but explicit open license
Software source code MUST be licensed under an OSI-approved open source license.	yes	LICENSE
All code MUST be published with a license file.	yes	LICENSE
Contributors MUST NOT be required to transfer copyright of their contributions to the codebase.	yes
All source code files in the codebase SHOULD include a copyright notice and a license header.	yes	2020-08-03 review by @ericherman
Codebases MAY have multiple licenses for different types of code and documentation.		N/A

Use a coherent style

• [x] compliant with this criterion.

Requirement	meets	links and notes
Contributions MUST adhere to either a coding or writing style guide, either your own or an existing one that is advertised in or part of the codebase.	yes	Style guides
Contributions SHOULD pass automated tests on style.	yes
Your codebase SHOULD include inline comments and documentation for non-trivial sections.	yes	add to contributing guidelines
You MAY include sections in your style guide on understandable English.

Document codebase maturity

• [x] compliant with this criterion.

Requirement	meets	links and notes
A codebase MUST be versioned.	yes	version list
A codebase that is ready to use MUST only depend on other codebases that are also ready to use.	yes	Open source dependencies
A codebase that is not yet ready to use MUST have one of these labels: prototype - to test the look and feel, and to internally prove the concept of the technical possibilities, alpha - to do guided tests with a limited set of users, beta - to open up testing to a larger section of the general public, for example to test if the codebase works at scale, pre-release version - code that is ready to be released but hasn’t received formal approval yet.	N/A	Is ready
A codebase SHOULD contain a log of changes from version to version, for example in the CHANGELOG.	yes	changelog

Open Source dependencies

Open Zaak would not be possible without a number of other Open Source dependencies.

Most notably, the backend is written in the Python programming language. We use the Django framework to build the web-application (which includes the API of course) that is Open Zaak.

The core elements for the API implementation are:

• Django REST framework (DRF), to implement the RESTful API

• drf-yasg to generate the API documentation in the form of OAS 3.0 specification

• VNG-API-common, a re-usable library with some specific VNG/Dutch API tooling, built on top of DRF and drf-yasg

What about the dependencies that don’t have a 1.0.0 version (yet)?

Good question!

Most libraries follow semantic versioning, which takes the form of A.B.c version numbering. In this pattern, A is the major version, B is the minor version and c is the patch version. Roughly speaking, if A increments, you can expect breaking changes. If B increments, the changes are backwards compatible fixes and new features, and if c changes, it’s purely a bugfix release.

Libraries with a version like 0.x.y are usually considering not-stable yet - as long as no 1.0.0 release has happened, the internal API can change, or the project may never reach that “maturity” you’d want.

If you look at our requirements, you will see a couple of libraries that don’t have a 1.0.0 version (yet). So, why do we depend on them, and is there a risk of depending on them? Below, you can find the mitigations/reasoning why we decided to depend on them anyway, in alphabetical order.

• cmislib-maykin==0.7.4 - this package is a fork of a Python 2 library implementing the CMIS bindings. Maykin Media (one of the Open Zaak service providers) ported it to Python 3 and maintains this fork.

• coreschema==0.0.4 is a transitive dependency of coreapi and drf-yasg, which are both well-maintained. It is made by the same author as DRF itself.

• dictdiffer==0.8.0 is not mission-critical and can easily be swapped out if needed. It’s currently used to visualize changes obtained from audit trail records in the admin interface. The library is Open Source on Github, and could be forked if needed.

• django-auth-adfs-db==0.2.0 is an add-on for django-auth-adfs. Maykin Media is one of the maintainers.

• django-db-logger==0.1.7 is a package authored and maintained by Maykin Media, and used in various other projects.

• django-extra-views==0.13.0 is a popular Django package with a large community behind it. The original author is still active as well.

• django-loose-fk==0.7.1 is a package authored by Maykin Media to serve Open Zaak. It can be considered an integral part of Open Zaak, but the code was separated out in a standalone package for easier maintenance.

• django-sendfile2==0.6.0 is a fork of django-sendfile, focusing on Python 3 support. The core functionality is very stable and unlikely to cause problems.

• djangorestframework-camel-case==0.2.0 is a fairly popular and standard DRF add-on. Schema generators like drf-yasg, drf-spectacular and DRF even have explicit support for this package.

• djangorestframework-gis==0.14 is used for the GeoJSON serialization. We only use a small set of features, but it’s the go-to GIS library in combination with DRF, and very stable despite the version numbering.

• drf-nested-routers==0.90.2 sees regular maintenance and activity on Github, with high popularity.

• drf-writable-nested==0.4.3 sees regular maintenance and activity on Github, with high popularity.

• gemma-zds-client==0.13.0 originally developed by Maykin Media for the demo’s of the VNG standard, this package is now used by multiple municipalities. VNG has been requested to transfer the package to Maykin Media, and 1.0 preparations are in progress at the time of writing.

• httplib2==0.19.0 transitive dependency of the CMIS binding. Recently saw a security release and is still maintained.

• inflection==0.3.1 transitive dependency of drf-yasg, which is quite a popular project.

• iso-639==0.4.5 stable package that just happens to never have been named 1.0. ISO-639 is an international standard, which don’t tend to change.

• iso8601==0.1.12 transitive dependency of the CMIS bindings. Appears to still be maintained.

• isodate==0.6.0 yet another library to parse ISO-8601 dates. Transitive dependency of vng-api-common.

• nlx-url-rewriter==0.1.2 no longer actively used, only here for historical reasons (migrations).

• python-dotenv==0.8.2 is a popular, actively maintained project on Github

• ruamel.yaml.clib==0.2.2 transitive dependency of ruamel.yaml, by the same author

• ruamel.yaml==0.16.7 transitive dependency of drf-yasg, actively maintained.

• sentry-sdk==0.16.5 SDK maintained by the Sentry.io team, it replaces raven. Sentry is widely adapted and has a large community of developers making use of the error-monitoring services through the SDK.

• sqlparse==0.3.0 direct dependency of Django. Given the widespreak use of Django, this should not pose any problems.

• zgw-consumers==0.12.2 library maintained by Maykin Media. Together with gemma-zds-client, preparations are in the works for a 1.0 version.

API-specifications

Open Zaak adheres to the API-specifications as described by the VNG standards for “API’s voor Zaakgericht werken”. The interaction between these API’s can be found there as well.

Supported API versions

The following API’s are available in Open Zaak:

API	Specification version(s)
Zaken API	1.0
Documenten API	1.0
Catalogi API	1.0
Besluiten API	1.0
Autorisaties API	1.0

In addition, Open Zaak requires access to a Notificaties API. Open Zaak uses Open Notificaties by default.

Installation

There are several ways to install Open Zaak. A scalable solution is to use Kubernetes. You can also run the Docker containers on a single machine.

Before you begin

• Check the minimum system requirements for the target machine(s).

• Make sure the target machine(s) have access to the Internet.

• The target machine(s) should be reachable via at least a local DNS entry:

  • Open Zaak: open-zaak.<organization.local>

  • Open Notificaties: open-notificaties.<organization.local>

  The machine(s) do not need to be publically accessible and do not need a public DNS entry. In some cases, you might want this but it’s not recommended. The same machine can be used for both Open Zaak and Open Notificaties.

• If you want to use NLX, make sure you have a publicaly available domain name, for example nlx.<organization.com>, where your NLX-inway is accessible to the outside world.

Guides

Hardware requirements

Based on our initial performance tests in both a Kubernetes environment and single machine setups, we can indicate some minimum system requirements to reach a certain performance.

A Kubernetes setup will scale more easy while being a little more expensive.

Determine what you need

Municipalities

For municipality users, we made a small table of to quickly look up how many users we expect to access the system concurrently based on the number of inhabitants. You can also directly look at the concurrent users in the performance table below if this is not applicable or you have a good idea of the number of concurrent users yourself.

Inhabitants	Expected concurrent users
10.000	100
50.000	250
100.000	500
500.000	1.000
1.000.000	2.000

Performance

Based on functional performance test scenario’s and requirements, we made a translation to technical performance requirements (requests per second). We used the following numbers to make the translation:

1. Average number of requests per functional scenario: 7 (higher means more requests per second)

2. Average waiting time between functional scenario’s: 5 minutes (lower means more requests per second)

Concurrent users	Expected requests per second
100	3
250	6
500	12
1.000	24
2.000	47

The above number for requests per second depends greatly on the the actual usage of the API. We used theoretical scenarios to give some indication, so it’s best to use a higher number when looking for the minimum system requirements below.

Minimum system requirements

• Platform: 64-bit

• Processor(s): 4 - 16 CPUs (see below) at 2.0 GHz

• RAM: 8 - 32 GB (see below)

• Hard disk space: 20 GB (excluding storage for documents)

Based on the number of requests per second you need, you can see what kind of hardware you need to achieve this.

Requests per second	CPUs	Memory (GB)
25	4	8
50	6	12
100	12	24
150	14	28
200	16	32

With these specifications you can run everything on a single machine or divided over several instances.

General recommendations

• Use a separate database server with roughly a third of the CPUs and memory as the main server. The database is usually the limiting factor.

Kubernetes recommendations

• Preferably use 2 load balancer (like Traefik) replica’s.

• Use as many replica’s as available CPU’s taking into account you need to have a few replica’s for your load balancer, and possibly other services.

Deploying on Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

—kubernetes.io

Kubernetes (K8s) is one of the main supported targets for deployment. We provide a fully automated deployment toolset suitable for DevOps engineers and/or system administrators.

This documentation will sketch out the architecture, document the initial requirements and teach you how to deploy on K8s.

This setup is tested against a Google Cloud Kubernetes cluster with 4vCPU and 15G of memory - two n2-standard-2 nodes. See the documentation for the available machine types. Please see Hardware requirements to determine your hardware requirements.

Note

Head’s up! If you’re familiar with Helm, there are community-provided Helm charts for Open Zaak and Open Notificaties!

Assumed/required knowledge level of Kubernetes

Kubernetes is a complex infrastructure with a lot of moving parts and a steep learning curve.

We assume you have some familiarity with the following concepts:

• Namespaces

• Deployments

• Services

• Ingresses

• Persistent volumes

Additionally, familiarity with the kubectl command line tool is a big advantage.

You should be able to save a kube-config file and optionally specify its location via the KUBECONFIG environment variable.

Target platform

This guide is written and tested against a Kubernetes cluster on Google Cloud.

Google cloud resources can be managed with the gcloud command line tool, or alternatively in the cloud console via point and click.

We have collected a number of example commands to spin up an environment.

Creating a cluster

See the documentation to create a cluster and the prerequisites. We assume a Google Cloud project name of gemeente.

[user@host]$ gcloud container --project "gemeente" clusters create "test-cluster" \
    --zone "europe-west4-b" \
    --cluster-version "1.14.8-gke.12" \
    --machine-type "n2-standard-2" \
    --num-nodes "2"

Creating a database

From the documentation to create an SQL instance:

[user@host]$ gcloud sql instances create test \
  --assign-ip \
  --region europe-west4 \
  --zone europe-west4-b \
  --database-version POSTGRES_11 \
  --root-password letmein \
  --cpu 2 \
  --memory 7GiB

In your Google Cloud console you can then configure the remote networks that are allowed to access this instance. You should at least include the network from where you are deploying.

Additionally, it’s recommended to set up a private IP so connections from the Kubernetes cluster are allowed.

Application architecture

Typically, your K8s cluster provider will set up a load balancer in front of your cluster, directing all the traffic to services in the cluster.

DNS is set up so that your domain name(s) point to the IP address of the load balancer.

The load balancer will then pass the traffic to an Ingress in the cluster, which will route the traffic to the backend services.

All applications make use of caching via Redis cache databases.

Open Zaak

Traffic from the ingress will enter Open Zaak at one or multiple NGINX reverse proxies. NGINX will performs one of two possible tasks:

• pass the request to the Open Zaak application or

• send the binary data of uploaded files to the client

NGINX exists to serve uploaded files in a secure and performant way, reducing load on the Open Zaak application.

All other requests are passed to the Open Zaak application.

Open Notificaties

Traffic from the ingress is directly passed to the Open Notificaties application. Open Notificaties then communicates with async workers (using RabbitMQ) to distribute the notifications to all the relevant subscribers.

Environment requirements

Before you begin, you will need:

• a kubernetes cluster that you can access, this means you need a valid ~/.kube/config file. You can override which kube config to use by setting the KUBECONFIG environment variable if you manage multiple clusters.

  If you are dealing with jump/bastion hosts, complicated firewalls… Please contact your provider on how you can access your cluster from your local machine.

• a PostgreSQL (10 or 11) database server with credentials:

  • a database hostname that you can reach from your local machine

  • a database hostname that can be reached from your K8s cluster (possibly the same as above)

  • the username of a superuser (typically postgres)

  • a password for the superuser

  • credentials for the Open Zaak database and Open Notificaties database

• A persistent-volume storage class supporting ReadWriteMany. Contact your provider to see if they offer it. If this is not an option, you can use a ReadWriteOnce storage class and set up an NFS-server around it, but this will likely have slower performance.

  On Google Cloud, you can use:

  [user@host]$ gcloud compute disks create --size=10GB --zone=europe-west4-b gce-nfs-disk
  

Deployment requirements

Fully automated deployment is implemented with Ansible. Ansible runs on your local machine (control host) and connects to the required services to realize the desired state.

For example, to create the application database, it will create a database connection and execute the necessary queries. To manage kubernetes objects, it will use the Kubernetes API via your KUBECONFIG.

Ansible is a Python tool and has a number of dependencies. The deployment is tested on Python 3.7.

Get a copy of the deployment configuration

You can either clone the https://github.com/open-zaak/open-zaak repository, or download and extract the latest ZIP: https://github.com/open-zaak/open-zaak/archive/master.zip

Ensure you have a suitable Python version

Check your operation system packages and make sure you have installed a recent enough Python version. We recommend using Python 3.7.

Create a virtual environment

Virtual environments isolate dependencies between environments. It gives us close control over the exact required versions.

Create a virtualenv:

[user@host]$ python3.7 -m venv env

This creates a virtualenv named env. Next, activate the virtualenv. You need to do this every time you want to use the deployment tooling.

[user@host]$ source env/bin/activate

Install dependencies

First, navigate to the correct directory. In the folder where you placed the copy of the repository, change into the deployment directory:

(env) [user@host]$ cd /path/to/open-zaak/deployment/

To install the required dependencies, we use the Python package manager pip:

Note

These requirements are different from the the ones in the Open Zaak root directory. They can be installed on top of eachother though.

(env) [user@host]$ pip install -r requirements.txt
(env) [user@host]$ cd kubernetes
(env) [user@host]$ ansible-galaxy collection install -r requirements.yml

Roughly said, this installs Ansible and the modules to talk to the PostgreSQL database and Kubernetes API.

Deploying (automated or manual)

Ansible has the concept of playbooks - a predefined set of tasks to execute, logically grouped.

Open Zaak ships with two playbooks:

• provision.yml:

  • finishes the configuration of your Kubernetes cluster (if needed)

  • initializes the application databases

• apps.yml:

  • installs Open Zaak

  • installs Open Notificaties

You can run the Ansible-playbooks as-is (with some configuration through variables), or use them an inspiration for manual deployment.

Provisioning

Below you find some guidance to modify the provisioning specifically to your needs.

I already have an ingress-controller

Set the variable needs_ingress in provision.yml to no. Otherwise, Traefik 2.0 is set up as Ingress controller.

I have a ReadWriteMany storage solution

Set the variable needs_nfs in provision.yml to no. Otherwise, a NFS-server is deployed to use as ReadWriteMany solution.

Todo

streamline nfs/RWX solution!

Database configuration

The playbook will set up the application database user(s) with the correct, minimal permissions and will set up the databases for the applications. To be able to do this, you need superuser access. See the vars/db_credentials.example.yml file for the example configuration.

Both Open Zaak and Open Notificaties require database configuration to be defined in the vars/open-zaak.yml and vars/open-notificaties.yml variable files:

openzaak_db_name: openzaak  # name of the database to create
openzaak_db_host: postgres.gemeente.nl  # hostname or IP address of the database server
openzaak_db_port: "5432"  # database server port, default is 5432
openzaak_db_username: openzaak  # username of the application database user
openzaak_db_password: secret  # password of the application database user

For Open Notificaties, the prefix is opennotificaties instead of openzaak.

Applying the provisioning

Run the provision.yml playbook using:

(env) [user@host]$ ./deploy.sh provision.yml

Applications

The apps.yml playbook sets up the Open Zaak and Open Notificaties installations.

I already have an ingress-controller

Todo

opt-out of the Traefik CRD and provide an alternative Ingress resource

Configuring Open Zaak

To deploy Open Zaak, some variables need to be set (in vars/open-zaak.yml):

• openzaak_domain: the domain name, e.g. open-zaak.gemeente.nl

• openzaak_secret_key: generate a key via https://miniwebtool.com/django-secret-key-generator/. Make sure to put the value between single quotes!

See roles/openzaak/defaults/main.yml for other possible variables to override. You might want to tweak environment variables in order to provision a superuser.

Configuring Open Notificaties

To deploy Open Notificaties, some variables need to be set (in vars/open-notificaties.yml):

• opennotificaties_domain: the domain name, e.g. open-notificaties.gemeente.nl

• opennotificaties_secret_key: generate a key via https://miniwebtool.com/django-secret-key-generator/. Make sure to put the value between single quotes!

See roles/opennotificaties/defaults/main.yml for other possible variables to override.

Configuring NLX

If you want to use NLX, you need to create NLX certificate. See https://docs.nlx.io/try-nlx/retrieve-a-demo-certificate. The key and certificate need to be set in vars/nlx.yml:

• nlx_inway_key: private key

• nlx_inway_cert: certificate

Deploying the applications

Make sure your kubectl command can access the correct cluster:

(env) [user@host]$ kubectl cluster-info

Run the apps.yml playbook using:

(env) [user@host]$ ./deploy.sh apps.yml

Next steps

You may want to customize the logging setup. The default setup should be sufficient to get started though.

To be able to work with Open Zaak, a couple of things have to be configured first, see Open Zaak configuration for more details.

Updating an Open Zaak installation

Make sure you have the deployment tooling installed - see Deployment requirements for more details.

If you have an existing environment (from the installation), make sure to update it:

# fetch the updates from Github
[user@host]$ git fetch origin

# checkout the tag of the version you wish to update to, e.g. 1.0.0
[user@host]$ git checkout X.Y.z

# activate the virtualenv
[user@host]$ source env/bin/activate

# ensure all (correct versions of the) dependencies are installed
(env) [user@host]$ pip install -r requirements.txt

Open Zaak deployment code defines variables to specify the Docker image tag to use. This is synchronized with the git tag you’re checking out.

Warning

Make sure you are aware of possible breaking changes or manual interventions by reading the Changelog!

Make sure your kubectl command can access the correct cluster:

(env) [user@host]$ kubectl cluster-info

Note

You need access to the cluster for kubectl to work. With Google Cloud you can get the proper configuration using the Google Cloud SDK:

(env) [user@host]$ gcloud container clusters get-credentials <cluster name> --region <region>

For the proper authentication setup, you might need a service account. See the [Getting Started with Authentication on Google Cloud](https://cloud.google.com/docs/authentication/getting-started#command-line).

Next, to perform the upgrade, you run the apps.yml playbook just like with the installation:

(env) [user@host]$ ./deploy.sh apps.yml

You can see the status and events with:

(env) [user@host]$ kubectl -n <namespace> get pods
(env) [user@host]$ kubectl -n <namespace> describe pod <pod name>

Note

In the Kubernetes deployment setup, Open Zaak makes use of multiple replicas by default, and is set up to perform rolling releases. This means that the old version stays live until all new versions are running without errors.

We make use of health checks and liveness probes to achieve this.

This does mean that there’s a brief window where clients may hit the old or new version at the same time - usually this shouldn’t pose a problem though.

Deploying on a single server

Open Zaak can be deployed on a single machine - either a dedicated server (DDS) or virtual private server (VPS). The required hardware can be rented from a hosting provider or be provided in your environment. Please see Hardware requirements to determine the hardware requirements.

This documentation describes the architecture, prerequisites and how to deploy Open Zaak on a server. Additionally, it documents the possible configuration options.

Note

The default settings allow Open Zaak to be deployed to the same machine as Open Notificaties.

Architecture

The application is deployed as Docker containers, of which the images are available on Docker hub. Traffic is routed to the server, where the web server (nginx) handles SSL termination and proxies the requests to the application containers.

Data is stored in a PostgreSQL database. By default, the database is installed on the same machine (running on the host), but you can make use of a hosted database (Google Cloud, AWS, Azure…). See the Configuration parameters for more information.

Prerequisites

Before you can deploy, you need:

A server

Ensure you have a server with root privileges. We assume you can directly ssh to the machine as root user. If that’s not the case, a user with sudo will also work. Python 3 must be available on the server.

Note

Make sure there is enough space in /var/lib/docker. You need at least 8 GB to download all Docker containers. We recommend placing the Docker folder wherever you also want to store your documents that are uploaded via the Documenten API.

Supported operating systems

Support for different Linux flavours is maintained in the Ansible collection used for deployment.

Currently the following OS flavours are supported:

• Debian: buster (10, actively supported), stretch (9, actively supported), jessie (8)

• Ubuntu: eoan (EOL), disco (EOL), cosmic (EOL), bionic (18.04 LTS). focal (20.04 LTS) is not tested yet

• SUSE Enterprise Linux: 15 (actively supported)

• OpenSUSE: 15.1

• Red Hat: 7, 8

• CentOS: 7, 8 (actively supported)

A copy of the deployment configuration

You can either clone the https://github.com/open-zaak/open-zaak repository, or download and extract the latest ZIP: https://github.com/open-zaak/open-zaak/archive/master.zip

Python and a Python virtualenv

You will need to have at least Python 3.5 installed on your system. In the examples, we assume you have Python 3.6.

Create a virtualenv with:

[user@laptop]$ python3.6 -m venv env/
[user@laptop]$ source env/bin/activate

Make sure to install the deployment tooling. In your virtualenv, install the dependencies:

(env) [user@laptop]$ pip install -r deployment/requirements.txt
(env) [user@laptop]$ cd deployment/single-server
(env) [user@laptop]$ ansible-galaxy collection install -r requirements.yml
(env) [user@laptop]$ ansible-galaxy role install -r requirements.yml

Deployment

Deployment is done with an Ansible playbook, performing the following steps:

1. Install and configure PostgreSQL database server

2. Install the Docker runtime

3. Install the SSL certificate with Let’s Encrypt

4. Setup Open Zaak with Docker

5. Install and configure nginx as reverse proxy

Initial steps

Make sure the virtualenv is activated:

[user@laptop]$ source env/bin/activate

Navigate to the correct deployment directory:

(env) [user@laptop]$ cd deployment/single-server

Create the vars/open-zaak.yml file - you can find an example in vars/open-zaak.yml.example. Generate a secret key using the Django secret key generator and put the value between single quotes.

Configure the host by creating the hosts file from the example:

(env) [user@laptop]$ cp hosts.example hosts

In the hosts file, edit the open-zaak.gemeente.nl to point to your actual domain name. You must make sure that the DNS entry for this domain points to the IP address of your server.

Warning

It’s important to use the correct domain name, as the SSL certificate will be generated for this domain and only this domain will be whitelisted by Open Zaak! If you are using a private DNS name, then no SSL certificate can be created via Letsencrypt - make sure to disable it by setting certbot_create_if_missing=false or openzaak_ssl=false if you don’t plan on using HTTPS at all.

Running the deployment

Execute the playbook by running:

(env) [user@laptop]$ ansible-playbook open-zaak.yml

Hint

• If you have your secrets Ansible vault encrypted, make sure you have either:

  • set the ANSIBLE_VAULT_PASSWORD_FILE environment variable, or

  • pass --ask-vault-pass flag to ansible-playbook.

• If you need to override any deployment variables (see Configuration parameters), you can pass variables to ansible-playbook using the syntax: --extra-vars "some_var=some_value other_var=other_value".

• If you want to run the deployment from the same machine as where it will run (ie. install to itself), you can pass --connection local to ansible-playbook.

• If you cannot connect as root to the target machine, you can pass --user <user> --become --become-method=sudo --ask-become-pass which will connect as user <user> that needs sudo-rights on the target machine to install the requirements.

A full example might look like this:

(env) [user@laptop]$ ansible-playbook open-zaak.yml \
    --user admin
    --inventory my-hosts \  # Use inventory file ``my-hosts`` instead of ``hosts``.
    --limit open-zaak.gemeente.nl \  # Only pick open-zaak.gemeente.nl from the inventory file.
    --extra-vars "openzaak_ssl=false openzaak_db_name=open-zaak-test openzaak_db_username=open-zaak-test" \
    --connection local \
    --become \
    --become-method=sudo \
    --ask-become-pass

Note

You can run the deployment multiple times, it will not affect the final outcome. If you decide to change configuration parameters, you do not have to start from scratch.

Changing environment variables

The Open Zaak configuration is templated out to /home/openzaak/.env on the host machine. It’s possible to modify environment variables here, but doing so will not become effective immediately - you need to restart the containers:

[root@host]# docker restart openzaak-0 openzaak-1 openzaak-2

Make sure to do this for every replica - you can see what’s running with docker ps.

Warning

If you modify the .env file and then apply the Ansible playbook again, this will overwrite your changes!

Environment configuration

After the initial deployment, some initial configuration is required. This configuration is stored in the database and is only needed once.

Create a superuser

A superuser allows you to perform all administrative tasks.

1. Log in to the server:

   [user@laptop]$ ssh root@open-zaak.gemeente.nl
   

2. Create the superuser (interactive on the shell). Note that the password you type in will not be visible - not even with asterisks. This is normal.

   This can also be automated, see Provisioning a superuser.

   [root@open-zaak.gemeente.nl]# docker exec -it openzaak-0 src/manage.py createsuperuser
   Gebruikersnaam: demo
   E-mailadres: admin@open-zaak.gemeente.nl
   Password:
   Password (again):
   Superuser created successfully.
   

Configure Open Zaak Admin

See the Open Zaak configuration on how to configure Open Zaak post-installation.

Configuration parameters

At deployment time, you can configure a number of parts of the deployment by overriding variables. You can override variables on the command line (using the -e "..." syntax) or by overriding them in vars/secrets.yml.

Note

Tweaking configuration parameters is considered advanced usage.

Generic variables

• certbot_admin_email: e-mail address to use to accept the Let’s Encrypt terms and conditions.

• openzaak_ssl: whether to use Let’s Encrypt to create an SSL certificate for your domain. Set to false if you want to use an existing certificate.

Open Zaak specific variables

The default values can be found in in the Ansible role.

• openzaak_db_port: database port. If you are running multiple PostgreSQL versions on the same machine, you’ll have to point to the correct port.

• openzaak_db_host: specify the hostname if you’re using a cloud database or a database on a different server.

• openzaak_db_name: specify a different database name.

• openzaak_db_username: specify a different database username.

• openzaak_db_password: specify a different database username.

• openzaak_secret_key: A Django secret key. Used for cryptographic operations - this may NOT leak, ever. If it does leak, change it.

Scaling

The openzaak_replicas variable controls scaling on backend services. If your hardware allows it, you can create more replicas. By default, 3 replicas are running.

The format of each replica is:

name: openzaak-i
port: 800i

The port number must be available on the host - i.e. you may not have other services already listening on that port.

Next steps

You may want to customize the logging setup. The default setup should be sufficient to get started though.

To be able to work with Open Zaak, a couple of things have to be configured first, see Open Zaak configuration for more details.

Updating an Open Zaak installation

Make sure you have the deployment tooling installed - see the installation steps for more details.

If you have an existing environment (from the installation), make sure to update it:

# fetch the updates from Github
[user@host]$ git fetch origin

# checkout the tag of the version you wish to update to, e.g. 1.0.0
[user@host]$ git checkout X.Y.z

# activate the virtualenv
[user@host]$ source env/bin/activate

# ensure all (correct versions of the) dependencies are installed
(env) [user@host]$ pip install -r requirements.txt
(env) [user@host]$ ansible-galaxy install -r requirements.yml

Open Zaak deployment code defines variables to specify the Docker image tag to use. This is synchronized with the git tag you’re checking out.

Warning

Make sure you are aware of possible breaking changes or manual interventions by reading the Changelog!

Next, to perform the upgrade, you run the open-zaak.yml playbook just like with the installation in Running the deployment:

(env) [user@laptop]$ ansible-playbook open-zaak.yml

Note

This will instruct the docker containers to restart using a new image. You may notice some brief downtime (order of seconds to minutes) while the new image is being downloaded and containers are being restarted.

Provisioning a superuser

A clean installation of Open Zaak has no existing data, it’s up to the service provider and/or the users to set up Open Zaak to your liking.

To be able to do anything in the Open Zaak admin interface, you need a user account with sufficient permissions, typically a superuser. Open Zaak has a couple of mechanisms to create this superuser.

Creating a superuser manually

Superusers can be created through the Command Line Interface (CLI) built into Open Zaak, for example:

python src/manage.py createinitialsuperuser \
    --username admin \
    --password admin \
    --email admin@gemeente.nl \
    --no-input

This will create the user if it does not exist yet. If the user already exists (based on username), nothing happens.

You can get detailed information by getting the built-in help:

python src/manage.py createinitialsuperuser --help

Note

Instead of providing the password as an argument, you can also use an environment variable DJANGO_SUPERUSER_PASSWORD or use the flag --generate-password. Generated passwords are e-mailed to the configured e-mail address.

Creating a superuser as part of the (initial) deployment

It’s possible to automatically provision a superuser as part of your regular deployment, be it on Kubernetes, Docker or Podman or any other way. Behind the scenes, the createinitialsuperuser management command is executed (see Command Line Interface (CLI) for more details).

To opt in to this behaviour, you must specify the following environment variables:

• OPENZAAK_SUPERUSER_USERNAME: username of the superuser account to create. If the account already exists, nothing will happen.

• OPENZAAK_SUPERUSER_EMAIL: e-mail address of the superuser account. Has a default value of admin@admin.org.

The DJANGO_SUPERUSER_PASSWORD environment variable is optional, if provided, this is the password that will be set for the superuser account.

The superuser will be created on container start-up, before the web-server starts.

Configuration

Open Zaak configuration

Before you can work with Open Zaak after installation, a few settings need to be configured first.

Setting the domain

In the admin, under Configuratie > Websites, make sure to change the existing Site to the domain under which Open Zaak will be deployed (see the manual for more information).

Configure Notificaties API

Next, the notifications for Open Zaak must be configured. We assume you’re also using Open Notificaties to make a complete setup.

There are 2 things to keep in mind:

1. Open Zaak offers an Autorisaties API and thus the Open Zaak Autorisaties API must be consulted to check for autorisations.

2. Each component handles authentication themselves and thus we need to store the Client IDs and secrets in each component that wants to communicate with eachother.

Open Zaak

1. Configure the Notificaties API endpoint (so Open Zaak knows where to send notifications to):

   1. Navigate to Configuratie > Notificatiescomponentconfiguratie

   2. Fill out the form:

      • API root: The URL to the Notificaties API. For example: https://open-notificaties.gemeente.local/api/v1/.

   3. Click Opslaan.

2. Configure the credentials for the Notificaties API (so Open Zaak can access the Notificaties API):

   1. Navigate to API Autorisaties > Services

   2. Click Service toevoegen.

   3. Fill out the form:

      • Label: For example: Open Notificaties

      • Type: Select the option: NRC (Notifications)

      • API root url: Same URL as used in step 1b.

      • Client ID: For example: open-zaak

      • Secret: Some random string. You will need this later on!

      • Authorization type: Select the option: ZGW client_id + secret

      • OAS: URL that points to the OAS, same URL as used in step 1b with /schema/openapi.yaml added to it for example: https://open-notificaties.gemeente.local/api/v1/schema/openapi.yaml

      • User ID: Same as the Client ID

      • User representation: For example: Open Zaak

   4. Click Opslaan.

3. Since Open Zaak also manages the authorizations via the Autorisaties API, we need to give Open Notificaties access to this API (so Open Notificaties can see who’s authorised to send notifications):

   1. Navigate to API Autorisaties > Applicaties

   2. Click Applicatie toevoegen.

   3. Fill out the form:

      • Label: For example: Open Notificaties

      • Client ID: For example: open-notificaties

      • Secret: Some random string. You will need this later on!

   4. Click Opslaan en opnieuw bewerken.

   5. Click Beheer autorisaties.

   6. Select first Component Autorisaties API and scope autorisaties.lezen.

   7. Select second Component Notificaties API and scope notificaties.consumeren.

   8. Click Opslaan

4. Finally, Open Notificaties will check if Open Zaak is allowed to send notifications (so we need to authorise Open Zaak for this):

   1. Navigate to API Autorisaties > Applicaties

   2. Click Applicatie toevoegen.

   3. Fill out the form:

      • Label: For example: Open Zaak

      • Client ID: The same Client ID as given in Open Zaak step 2c

      • Secret: The same Secret as given in Open Zaak step 2c

   4. Click Opslaan en opnieuw bewerken.

   5. Click Beheer autorisaties.

   6. Select component Notificaties API and scopes notificaties.consumeren and notificaties.publiceren.

   7. Click Opslaan

Currently, Open Zaak does not require any webhook subscriptions. It will however send notifications on various API actions.

We’re not there yet! We need to configure Open Notificaties as well!

Open Notificaties

1. Configure the Open Zaak Autorisaties API endpoint (so Open Notificaties knows where to check for the proper autorisations):

   1. Navigate to Configuratie > Autorisatiecomponentconfiguratie

   2. Fill out the form:

      • API root: The URL to the Autorisaties API. For example: https://open-zaak.gemeente.local/autorisaties/api/v1/.

      • Component: Notificatierouteringscomponent

   3. Click Opslaan.

2. Configure the Open Notificaties Notificatiescomponent API endpoint (so Open Notificaties receives changes made in the autorisation component of Open Zaak ):

   1. Navigate to Configuratie > Notificatiescomponentconfiguratie

   2. Fill out the form:

      • API root: The URL to the Notificaties API. For example: https://open-notificaties.gemeente.local/api/v1/.

   3. Click Opslaan.

   4. Webhook subscription toevoegen:

      • Callback Url: The Callback URL to the Notificaties Callback API. For example: https://open-notificaties.gemeente.local/api/v1/callbacks.

      • Client ID: The same Client ID as given in Open Zaak step 3c

      • Client Secret: The same Secret as given in Open Zaak step 3c

      • Channels: autorisaties

3. Configure the credentials for the Open Zaak Autorisaties API (so Open Notificaties can access the Autorisaties API):

   1. Navigate to API Autorisaties > Externe API credentials

   2. Click Externe API credential toevoegen.

   3. Fill out the form:

      • API root: Same URL as used in step 1b.

      • Label: For example: Open Zaak

      • Client ID: The same Client ID as given in Open Zaak step 3c

      • Secret: The same Secret as given in Open Zaak step 3c

      • User ID: Same as the Client ID

      • User representation: For example: Open Notificaties

   4. Click Opslaan.

4. Configure the credentials for the Open Notificaties API (so Open Notificaties can access itself):

   1. Navigate to API Autorisaties > Externe API credentials

   2. Click Externe API credential toevoegen.

   3. Fill out the form:

      • API root: The URL to the Notificaties API. For example: https://open-notificaties.gemeente.local/api/v1/.

      • Label: For example: Eigen API

      • Client ID: The same Client ID as given in Open Zaak step 3c

      • Secret: The same Secret as given in Open Zaak step 3c

      • User ID: Same as the Client ID

      • User representation: For example: Open Notificaties

   4. Click Opslaan.

5. We need to allow Open Zaak to access Open Notificaties (for authentication purposes, so we can then check its authorisations):

   1. Navigate to API Autorisaties > Client credentials

   2. Click Client credential toevoegen.

   3. Fill out the form:

      • Client ID: The same Client ID as given in Open Zaak step 2c

      • Secret: The same Secret as given in Open Zaak step 2c

   4. Click Opslaan.

6. Finally, we need to allow Open Notificaties to access Open Notificaties (for notifications purposes, so we can receive notificaties):

   1. Navigate to API Autorisaties > Client credentials

   2. Click Client credential toevoegen.

   3. Fill out the form:

      • Client ID: The same Client ID as given in Open Zaak step 3c

      • Secret: The same Secret as given in Open Zaak step 3c

   4. Click Opslaan.

All done!

Register notification channels

Before notifications can be sent to kanalen in Open Notificaties, these kanalen must first be registered via Open Zaak.

Register the required channels:

$ python src/manage.py register_kanaal zaken
$ python src/manage.py register_kanaal documenten
$ python src/manage.py register_kanaal besluiten
$ python src/manage.py register_kanaal autorisaties
$ python src/manage.py register_kanaal zaaktypen
$ python src/manage.py register_kanaal informatieobjecttypen
$ python src/manage.py register_kanaal besluittypen

Create an API token

By creating an API token, we can perform an API test call to verify the succesful installation.

Navigate to API Autorisaties > Applicaties and click on Applicatie toevoegen in the top right.

Give the application a label, such as test or demo, and fill out a demo client ID and secret. Next, click on Opslaan en opnieuw bewerken in the bottom right. The application will be saved and you will see the same page again. Now, click on Beheer autorisaties in the bottom right, which brings you to the authorization management for this application.

1. Select Catalogi API for the Component field

2. Check the catalogi.lezen checkbox

3. Click Opslaan in the bottom right

On the application detail page, you can now select and copy the JSON Web Token (JWT) shown under Client credentials, which is required to make an API call.

Warning

The JWT displayed here expires after a short time (1 hour by default) and should not be used in real applications. Applictions should use the client ID and secret pair to generate JWT’s on the fly.

Making an API call

We can now make an HTTP request to one of the APIs of Open Zaak. For this example, we have used Postman to make the request.

Make sure to set the value of the Authorization header to the JWT that was copied in the previous step.

Then perform a GET request to the list display of ZaakTypen (Catalogi API) - this endpoint is accessible at {{base_url}}/catalogi/api/v1/zaaktypen (where {{base_url}} is set to the domain configured in Setting the domain).

A GET request to the Catalogi API using Postman

Environment configuration reference

Open Zaak can be ran both as a Docker container or directly on a VPS or dedicated server. It relies on other services, such as database and cache backends, which can be configured through environment variables.

Available environment variables

Required

• DJANGO_SETTINGS_MODULE: which environment settings to use. Available options:

  • openzaak.conf.production

  • openzaak.conf.staging

  • openzaak.conf.docker

  • openzaak.conf.dev

  • openzaak.conf.ci

• SECRET_KEY: secret key that’s used for certain cryptographic utilities. You should generate one via miniwebtool

• ALLOWED_HOSTS: a comma separated (without spaces!) list of domains that serve the installation. Used to protect against Host header attacks.

Docker

Additionally, the following optional envvars MUST be set/changed when running on Docker, since localhost is contained within the container:

• CACHE_DEFAULT

• CACHE_AXES

• EMAIL_HOST

Optional

• SITE_ID: defaults to 1. The database ID of the site object. You usually won’t have to touch this.

• DEBUG: defaults to False. Only set this to True on a local development environment. Various other security settings are derived from this setting!

• IS_HTTPS: defaults to the inverse of DEBUG. Used to construct absolute URLs and controls a variety of security settings.

• DB_HOST: hostname of the PostgreSQL database. Defaults to localhost, unless you’re using the docker environment, then it defaults to db.

• DB_USER: username of the database user. Defaults to openzaak, unless you’re using the docker environment, then it defaults to postgres.

• DB_PASSWORD: password of the database user. Defaults to openzaak, unless you’re using the docker environment, then it defaults to no password.

• DB_NAME: name of the PostgreSQL database. Defaults to openzaak, unless you’re using the docker environment, then it defaults to postgres.

• DB_PORT: port number of the database, defaults to 5432.

• USE_X_FORWARDED_HOST: whether to grab the domain/host from the X-Forwarded-Host header or not. This header is typically set by reverse proxies (such as nginx, traefik, Apache…). Default False - this is a header that can be spoofed and you need to ensure you control it before enabling this.

• CACHE_DEFAULT: redis cache address for the default cache. Defaults to localhost:6379/0.

• CACHE_AXES: redis cache address for the brute force login protection cache. Defaults to localhost:6379/0.

• EMAIL_HOST: hostname for the outgoing e-mail server. Defaults to localhost.

• EMAIL_PORT: port number of the outgoing e-mail server. Defaults to 25. Note that if you’re on Google Cloud, sending e-mail via port 25 is completely blocked and you should use 487 for TLS.

• EMAIL_HOST_USER: username to connect to the mail server. Default empty.

• EMAIL_HOST_PASSWORD: password to connect to the mail server. Default empty.

• EMAIL_USE_TLS: whether to use TLS or not to connect to the mail server. Defaults to False. Should be True if you’re changing the EMAIL_PORT to 487.

• MIN_UPLOAD_SIZE: the max allowed size of POST bodies, in bytes. Defaults to 4GB. Note that you should also configure your web server to allow this.

• SENDFILE_BACKEND: which backend to use for authorization-secured upload downloads. Defaults to sendfile.backends.nginx. See django-sendfile2 for available backends.

• SENTRY_DSN: URL of the sentry project to send error reports to. Default empty, i.e. -> no monitoring set up. Highly recommended to configure this.

• JWT_EXPIRY: duration a JWT is considered to be valid, in seconds. Defaults to 3600 - 1 hour.

• LOG_STDOUT: whether to log to stdout or not. For Docker environments, defaults to True, for other environments the default is to log to file.

• PROFILE: whether to enable profiling-tooling or not. Applies to the development settings only. Defaults to False.

• CMIS_ENABLED: whether to enable the CMIS adapter. Defaults to False.

• CMIS_MAPPER_FILE: name of the file containing the mapping between the Django and Document Management System names for document properties. See the installation section for more details. Defaults to the absolute path of open-zaak/config/cmis_mapper.json.

• CMIS_URL_MAPPING_ENABLED: enable the URL shortener when using the CMIS adapter. Defaults to False.

• EXTRA_VERIFY_CERTS: a comma-separated list of paths to certificates to trust, empty by default. If you’re using self-signed certificates for the services that Open Zaak communicates with, specify the path to those (root) certificates here, rather than disabling SSL certificate verification. Example: EXTRA_VERIFY_CERTS=/etc/ssl/root1.crt,/etc/ssl/root2.crt.

• CURL_CA_BUNDLE: if this variable is set to an empty string, it disables SSL/TLS certificate verification. More information about why can be found here. Even calls from Open Zaak to other services such as the Selectie Lijst will be disabled, so this variable should be used with care to prevent unwanted side-effects.

Initial superuser creation

A clean installation of Open Zaak comes without pre-installed or pre-configured admin user by default.

Users of Open Zaak can opt-in to provision an initial superuser via environment variables. The user will only be created if it doesn’t exist yet.

• OPENZAAK_SUPERUSER_USERNAME: specify the username of the superuser to create. Setting this to a non-empty value will enable the creation of the superuser. Default empty.

• OPENZAAK_SUPERUSER_EMAIL: specify the e-mail address to configure for the superuser. Defaults to admin@admin.org. Only has an effect if OPENZAAK_SUPERUSER_USERNAME is set.

• DJANGO_SUPERUSER_PASSWORD: specify the password for the superuser. Default empty, which means the superuser will be created without password. Only has an effect if OPENZAAK_SUPERUSER_USERNAME is set.

Cross-Origin-Resource-Sharing

The following parameters control the CORS policy.

• CORS_ALLOW_ALL_ORIGINS: allow cross-domain access from any client. Defaults to False.

• CORS_ALLOWED_ORIGINS: explicitly list the allowed origins for cross-domain requests. Defaults to an empty list. Example: http://localhost:3000,https://some-app.gemeente.nl.

• CORS_ALLOWED_ORIGIN_REGEXES: same as CORS_ALLOWED_ORIGINS, but supports regular expressions.

• CORS_EXTRA_ALLOW_HEADERS: headers that are allowed to be sent as part of the cross-domain request. By default, Authorization, Accept-Crs and Content-Crs are already included. The value of this variable is added to these already included headers. Defaults to an empty list.

Specifying the environment variables

There are two strategies to specify the environment variables:

• provide them in a .env file

• start the Open Zaak processes (with uwsgi/gunicorn/celery) in a process manager that defines the environment variables

Providing a .env file

This is the most simple setup and easiest to debug. The .env file must be at the root of the project - i.e. on the same level as the src directory ( NOT in the src directory).

The syntax is key-value:

SOME_VAR=some_value
OTHER_VAR="quoted_value"

Provide the envvars via the process manager

If you use a process manager (such as supervisor/systemd), use their techniques to define the envvars. The Open Zaak implementation will pick them up out of the box.

CMIS adapter

In a default installation of Open Zaak, any documents created through the Documenten API are stored on disk and their metadata is stored in the database. However, it is also possible to store these documents in a Document Management System (DMS) using the CMIS standard.

The CMIS adapter converts API calls to the Documenten API in Open Zaak, to CMIS calls which are sent to the DMS to retrieve, create, update and delete documents. This way, the documents are stored in, or retrieved from, the DMS and not in or from Open Zaak.

CMIS support

CMIS 1.0 and CMIS 1.1 have various CMIS protocol bindings that can be used. Although according to the CMIS specification repositories must implement Web Services and AtomPub bindings, some DMS implementation only support one and/or recommend the newer Browser bindings.

	CMIS 1.0	CMIS 1.1
Web Services binding	Supported	Untested
AtomPub binding	Untested	Untested
Browser binding	N/A	Supported

CMIS support is built in Open Zaak using the CMIS adapter library. For an up-to-date list of supported CMIS versions and libraries, please see this project’s documentation.

Warning

The CMIS adapter is currently an experimental feature. While we have extensive unit test coverage with Alfresco, we require more “real world” testing before we can label the feature as stable.

Using the CMIS adapter

1. Create a mapping file to match Documenten API attributes to custom properties in your DMS model. The format is explained in the CMIS adapter library Mapping configuration documentation.

   You can use our default CMIS mapping for inspiration or just use these as defaults.

2. Make sure the content model is loaded in your DMS and matches the CMIS mapping described in step 1. It’s important that all attributes are present. Some need to be indexed to allow the proper CMIS queries to be executed.

   You can use our Alfresco model that matches the default mapping. The detailed explanation is described in the CMIS adapter library DMS Content model configuration documentation.

3. Enable the CMIS adapter. In the environment (or .env file), add or update the variable CMIS_ENABLED and CMIS_MAPPER_FILE:

   # Enables the CMIS-backend and the Open Zaak admin interface for configuring
   # the DMS settings.
   CMIS_ENABLED = True
   
   # Absolute path to the mapping of Documenten API attributes to (custom)
   # properties in your DMS model.
   CMIS_MAPPER_FILE = /path/to/cmis_mapper.json
   

4. Optional: enable URL mapping. In DMSs such as Corsa, queryable text fields can only be up to 100 characters long. However, documents in the Document API have URLs associated with it that are longer. The URL mapper feature shrinks the URLs so that they fit in queryable text fields.

   In the environment (or .env file), add or update the variable CMIS_URL_MAPPING_ENABLED:

   # Enables URL mapping in the CMIS-backend so that the URLs saved in
   # the DMS are shorter than 100 chars.
   CMIS_URL_MAPPING_ENABLED = True
   

5. You will need to restart Open Zaak for these changes to take effect.

6. Login to the Open Zaak admin interface (/admin/) as superuser.

7. Navigate to Configuratie > CMIS configuration and fill in all relevant fields.

8. If the URL mapping feature was enabled (point 4. above), then the mappings between the original and short version of a URL need to be defined. In the section URL MAPPINGS, fill in the field LONG PATTERN with the original URL (in format https://<domain>[/subpath]) and the SHORT PATTERN with the URL with shortened domain and subpath (in format https://<short domain>). The short pattern field can be at most 15 characters. The scheme (http or https) should be the same for both the long and short pattern.

   Warning

   Once a mapping has been defined, it cannot be updated or deleted.

9. Save the configuration with Opslaan en opnieuw bewerken.

10. You will see the CMIS connection status shows OK if everything went well.

Additional notes on creating documents

Depending on whether the CMIS adapter is enabled, there is a difference in behaviour for creating documents with an empty identification field.

If the CMIS adapter is disabled, the procedure to automatically generate the identification is as follows:

1. The prefix DOCUMENT is combined with the year of creation of the document. For example: DOCUMENT-2020-

2. All existing documents are searched to find all those with an identification field that starts with the generated prefix. These would for example be DOCUMENT-2020-0000000001, DOCUMENT-2020-0000000002, DOCUMENT-2020-0000000003.

3. The new document is given an identification field with a unique number that is different from those of all the other documents. This would for example be DOCUMENT-2020-0000000004.

The search done in point 2. requires an SQL LIKE clause, which is not supported by all DMSs. For this reason, if the CMIS adapter is in use, the automatically generated identification field will be equal to the document UUID.

Using self-signed certificates

Open Zaak supports self-signed certificates in two ways:

• Hosting Open Zaak using self-signed certificates - this is the classic route where your web server/ingress is configured appropriately

• Consuming services hosted with self-signed certificates - this is what this guide is about.

Note

Some background!

Open Zaak communicates with external services such as Open Notificaties, Github and the VNG Selectielijst service. It does this using https - using http is insecure.

When Open Zaak makes these requests, the SSL certificates are varified for their validity - e.g. expired certificates or certificates signed by an unkonwn Certificate Authority (CA) will throw errors (as they should!).

When you’re using self-signed certificates, you are essentially using an unkonwn CA, and this breaks the functionality of Open Zaak.

Adding your own certificates or CA (root) certificate

Open Zaak supports adding extra, custom certificates to the provided CA bundle. You do this by setting an environment variable EXTRA_VERIFY_CERTS, which must be a comma-separated list of paths to certificate files in PEM format.

An example of such a certificate is:

-----BEGIN CERTIFICATE-----
MIIByDCCAW8CFBRCXMlcdJAPb8XkG4cYMNL+Ku17MAoGCCqGSM49BAMCMGcxCzAJ
BgNVBAYTAk5MMRYwFAYDVQQIDA1Ob29yZC1Ib2xsYW5kMRIwEAYDVQQHDAlBbXN0
ZXJkYW0xEjAQBgNVBAoMCU9wZW4gWmFhazEYMBYGA1UEAwwPT3BlbiBaYWFrIFRl
c3RzMB4XDTIxMDMxOTExMDYyM1oXDTI0MDMxODExMDYyM1owZzELMAkGA1UEBhMC
TkwxFjAUBgNVBAgMDU5vb3JkLUhvbGxhbmQxEjAQBgNVBAcMCUFtc3RlcmRhbTES
MBAGA1UECgwJT3BlbiBaYWFrMRgwFgYDVQQDDA9PcGVuIFphYWsgVGVzdHMwWTAT
BgcqhkjOPQIBBggqhkjOPQMBBwNCAASzDq7C9atfN3uxoAGOCro8RfzWloVusDeO
bwXztxUC/wBu4WgfRsYjg65eVzaJWQKvIKn5W9rGyuIAYbJZJtMZMAoGCCqGSM49
BAMCA0cAMEQCIHKCp4qVEzF3WgaL6jY4tf60HBThnQTaXC99P7TaIFhxAiASMBVV
tmukm/NP8zSMrNpEGLnGIFa8uU/d8VwNNPFhtA==
-----END CERTIFICATE-----

Typically you would do this by (bind) mounting a volume in the Open Zaak container containing these certificates, and then specify their paths in the container, for example:

docker run \
    -it \
    -v /etc/ssl/certs:/certs:ro \
    -e EXTRA_VERIFY_CERTS=/certs/root1.crt,/certs/root2.crt
    open-zaak/open-zaak:latest

Of course, you will need to adapt this solution to your deployment method (Helm, Kubernetes, single-server…).

Post-install checklist

After Open Zaak has been installed successfully, go through the following checklist to see if the software works as expected:

Check configuration

Check the configuration page for Open Zaak, accessible at the url https://open-zaak.gemeente.nl/view-config/. This page will indicate whether certain settings are properly configured:

Run check management commands

To verify that notifications can be received by the URL as defined in the configuration, send a test notification using the following command:

python src/manage.py send_test_notification

Note

The notification is sent to the test channel - make sure it exists in the Notificaties API you’re using!

Run the Postman collection

Finally, run a simple Postman collection on the new install of Open Zaak:

• Create an Applicatie with superuser permissions via the admin

• A simple Postman collection has been published here, click on the Run in Postman button in the top right.

• Once the collection is loaded into Postman, click the icon in the top right to manage environments:

• Then click Add, enter the correct environment variables, then click Update:

• Click on the Runner button in the top left of the screen, then select the imported collection and the created environment and click Run

• If Open Zaak is properly configured, the tests will pass

Updating an Open Zaak installation

Software receives new features and bugfixes all the time, and Open Zaak is no different. At some point, you’ll want to update to a newer version of Open Zaak - be it a patch version with bugfixes or a feature release to upgrade to.

Before you actually upgrade, we strongly advise you to take a look at the Changelog to spot any breaking changes or required manual interventions.

Note

We always recommend you to have taken and tested your backups in case something goes wrong, BEFORE performing any updates.

Note

We encourage having multiple environments such as staging and production, completely isolated from each other. This allows you to test out the update on a staging environment to check if anything goes wrong, without affecting production.

The update instructions are split per target environment.

Updating on Kubernetes

See Updating an Open Zaak installation for the update instructions on Kubernetes.

Updating a single server installation

See the steps on how to update a single server installation.

Extra

Command Line Interface (CLI)

Open Zaak ships with a command line interface accessible inside of the containers.

To get more information about a command, you can run:

$ python src/manage.py <command> --help

Available commands

createinitialsuperuser

Creates an initial superuser with the specified username and e-mail address.

The password can be provided upfront with the --password CLI argument, or by using the DJANGO_SUPERUSER_PASSWORD environment variable. Additionally, with --generate-password a password can be generated and e-mailed to the specified e-mail address. Note that this requires your e-mail configuration to be set up correctly (any of the EMAIL_* envvars)!

setup_configuration

A CLI alternative to the point-and-click configuration in the admin interface.

send_test_notification

After configuring the Notificaties API, send a test nofification to verify the setup.

register_kanaal

Registers a notifications channel with the notifications API if it doesn’t exist yet. Channels must exist before Open Zaak can publish notifications to them.

Logging

Logging is a valuable tool to discover and debug issues happening when using Open Zaak.

Issues can be of different nature, and a different logging approach is suitable to those. For example, the following types of events are logged in some form or another:

• unexpected application errors, as a result of programming mistakes (bugs) - these logs are technical in nature and should be accessible for Open Zaak developers.

• application server logs - startup and access logs of the server running Open Zaak. Useful to see if client requests actually reach the ‘backend’

• webserver logs - Open Zaak uses a complex set-up to serve user-uploaded files securily. Things can go wrong here too.

• container logs - the docker container itself may have problems that need to be investigated.

This guide walks you through the various options on how to access and configure the logs.

Monitoring

Sentry focuses on tracking down errors in software, i.e. the Open Zaak application. We strongly recommend setting up this integration.

Open Zaak has support to itegrate with Sentry error monitoring. Whenever a bug occurs in Open Zaak, the client will receive an error response and the technical details of the error are sent to the Sentry project, with context.

Note

Sentry integration makes sure to strip sensitive context from technical details. Passwords and/or other credentials are not sent to Sentry, if they happen to be in the request context.

For documentation on how to set up a project in Sentry, please refer to the official documentation (make sure to follow the instructions for the platform Python > Django).

After setting up the project, you will receive a DSN, which is the URL to which exceptions will be sent (e.g. https://e95a42137e6042c59d19376e566f027a@sentry.openzaak.nl/104).

The created Sentry project can be linked to Open Zaak by setting the environment variable SENTRY_DSN equal to this DSN.

Viewing nginx logs

Nginx is the webserver sitting between the client and the Open Zaak backend. It mostly proxies requests to the backend, but it takes care of serving Documenten API files after the authorization checks are performed.

On Kubernetes

Many Kubernetes providers provide a graphical interface to view logs, for example on Google Cloud:

1. Navigate to your Kubernetes cluster

2. Via Workloads, find the deployment nginx

3. Find and click the Container logs link

Or, via the CLI tool kubectl:

# for convenience, set up the k8s namespace
[user@host]$ kubectl config set-context --current --namespace=openzaak-test

# fetch the logs
[user@host]$ kubectl logs -l app.kubernetes.io/name=nginx

# or for a single pod:
[user@host]$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
cache-79455b996-62llx       1/1     Running   0          68d
nginx-8579d9dfbd-8dn5m      1/1     Running   0          7h3m
nginx-8579d9dfbd-h4tc4      1/1     Running   0          7h3m
openzaak-59df44f556-7znvg   1/1     Running   0          7h2m
openzaak-59df44f556-gb4lq   1/1     Running   0          7h3m
openzaak-59df44f556-nqtr2   1/1     Running   0          7h3m

[user@host]$ kubectl logs --since=24h nginx-8579d9dfbd-8dn5m

On a VMWare appliance or single-server

On a single-server setup, nginx is not containerized and the log files can be found in /var/log/nginx:

• /var/log/nginx/error.log contains errors encountered by nginx

• /var/log/nginx/access.log is the access log of all the client requests

Application server, application and container logs

The application server, the application and the container itself write logs (together they make up the ‘backend’).

1. When the container starts up, it performs some checks before it proceeds with the application server startup.

2. Then, the application server starts up and writes some status information.

3. Every request that is received by the application server is logged as well - this is the access log.

4. Finally, the application itself detects potential problematic situations or writes other informational messages to the logging output.

All of these logs are logged to the container logs.

Viewing the container logs on Kubernetes

As with the nginx logs on Kubernetes, you can make use of your provider’s graphical interface if available.

Otherwise, you can use the CLI tool:

# for convenience, set up the k8s namespace
[user@host]$ kubectl config set-context --current --namespace=openzaak-test

# fetch the logs
[user@host]$ kubectl logs -l app.kubernetes.io/name=django

# or for a single pod:
[user@host]$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
cache-79455b996-62llx       1/1     Running   0          68d
nginx-8579d9dfbd-8dn5m      1/1     Running   0          7h3m
nginx-8579d9dfbd-h4tc4      1/1     Running   0          7h3m
openzaak-59df44f556-7znvg   1/1     Running   0          7h2m
openzaak-59df44f556-gb4lq   1/1     Running   0          7h3m
openzaak-59df44f556-nqtr2   1/1     Running   0          7h3m

[user@host]$ kubectl logs --since=24h openzaak-59df44f556-gb4lq

On a VMWare appliance or single-server

Unfortunately, docker does not seem to be able to aggregate logs from different containers. This means that if you are running multiple replicas of Open Zaak (which is the default), you may have to dig around a bit before you find what you are looking for.

To view the logs of a particular replica:

# first replica
[root@server]# docker logs openzaak-0

# second replica
[root@server]# docker logs openzaak-1

Check the Docker documentation for more information about logs in Docker.

Customizing the log output

Logging to file instead

By default, we configure Open Zaak to log to stdout in containers by setting the environment variable LOG_STDOUT=1.

You may wish to log to files instead, by using persistent volumes. If you decide to do this, then:

1. Make sure to mount the volume on /app/log - this is where log files are written to.

2. When multiple replicas are used, the volume must be ReadWriteMany on Kubernetes.

3. Set the environment variable LOG_STDOUT=0 to fall back to file-based logging.

Note

Log files are by default rotated - once a log file reaches 10MB, a new file is created and once 10 files exist, the oldest is deleted.

Logging infrastructure

Various log-aggregation solutions exist in the industry, such as ELK Stack, Grafana, fluentd and others. Consult their documentation on how to integrate them with Docker and/or Kubernetes.

Different needs?

Talk to us on Github if the current infrastructure does not fit your needs!

Manual

Note

The manual is currently only available in the Dutch language!

Once Open Zaak is installed, it needs to be configured to make all API’s accessible and allow inter-communication.

In addition, you can manage the data that is exposed in the Catalogi API directly in the management interface.

Algemene onderwerpen

De algemene onderwerpen beschrijven acties voor medewerkers met toegang tot de beheerinterface van Open Zaak (hierna de admin genoemd).

In deze handleiding nemen we aan dat Open Zaak geïnstalleerd is en beschikbaar op het adres https://open-zaak.gemeente.nl.

Inloggen

Om in te loggen in de admin kan je navigeren naar de startpagina https://open-zaak.gemeente.nl. Klik vervolgens op Beheer:

Vul je gebruikersnaam en wachtwoord in op het loginscherm:

Na het aanmelden zie je het dashboard. Afhankelijk van je gebruikersrechten zie je meer of minder items op het dashboard.

Wachtwoord wijzigen

Eenmaal ingelogd, kan je je wachtwoord wijzigen via de link rechtsboven:

Vul vervolgens je huidige wachtwoord in, je nieuwe wachtwoord en je nieuwe wachtwoord ter bevestiging.

Klik rechtsonderin op Mijn wachtwoord wijzigen om je nieuwe wachtwoord in te stellen.

Note

Merk op dat er bepaalde regels gelden om een voldoende sterk wachtwoord in te stellen. We raden aan om een password manager te gebruiken om een voldoende sterk wachtwoord in te stellen.

Dashboard

De gegevens die in de admin beheerd kunnen worden, zijn gegroepeerd op het dashboard. Deze groepen worden hier verder beschreven. Merk op dat het mogelijk is dat je bepaalde groepen niet ziet omdat je onvoldoende rechten hebt.

Accounts

Gebruikers zijn de personen die in kunnen loggen in de admin. Aan gebruikers worden rechten toegekend die bepalen wat ze precies kunnen inzien en/of beheren. Gebruikers kunnen gedeactiveerd worden, waardoor ze niet langer in kunnen loggen. Ga naar Een gebruiker aanmaken (en aan een groep toevoegen) om te leren hoe je een gebruiker toevoegt en configureert.

Groepen definiëren een set van permissies die een gebruiker toelaten om gegevens in te zien en/of beheren. Een gebruiker kan tot één of meerdere groepen behoren. Lees meer over groepen beheren.

API Autorisaties

De API’s voor zaakgericht werken zijn niet toegankelijk zonder autorisatie. Dit betekent dat elke applicatie die gegevens ophaalt of registreert in Open Zaak hiervoor moet geautoriseerd zijn. We spreken van taakapplicaties.

Via Applicaties wordt elke taakapplicatie geconfigureerd om de toegang in te regelen. Zie Een Applicatie registreren voor hoe je dit doet.

Daarnaast maakt Open Zaak zelf gebruik van andere API’s - bijvoorbeeld de Notificaties API, maar ook externe catalogi, Zaken API’s… zijn mogelijk. Via Externe API credentials kan je instellen hoe Open Zaak zichzelf bij deze API’s autoriseert.

Gegevens

De groep gegevens laat je toe om gevens in te kijken die via de Open Zaak API’s aangemaakt en/of gewijzigd worden.

Besluiten toont de besluiten die ontsloten worden via de Besluiten API.

Catalogi laat je toe om de (zaaktype)catalogi te beheren. Hoe je dit doet, is uitgebreid gedocumenteerd in Catalogusbeheer. De gegevens worden ontsloten met de Catalogi API.

Documenten laat je toe om informatieobjecten en gerelateerde objecten in te kijken en beheren. Deze worden via de Documenten API aangemaakt en gewijzigd.

Zaken bevat alle informatie die de Zaken API ontsluit. Hier kan je volledige zaakdossiers inkijken.

Configuratie

Het configuratiegedeelte dient om de Open Zaak-installatie te configureren. Typisch wordt dit initieel bij installatie geconfigureerd.

Via Access attempts en Access logs kan je de inlogpogingen en sessies in de admin van gebruikers bekijken. Deze worden gelogd om brute-forcing tegen te kunnen gaan en inzicht te verschaffen in wie op welk moment toegang had tot het systeem.

In de Notificatiescomponentconfiguratie kan je instellen van welke Notificaties API je gebruik maakt. Je moet een geldige configuratie instellen, anders worden er door Open Zaak geen notificaties verstuurd.

Webhook subscriptions bevat de abonnementen die Open Zaak afneemt bij de Notificaties API. Indien je geen gebruik maakt van de Autorisaties API van Open Zaak, maar een externe API, dan moet je een abonnement afnemen op het autorisaties kanaal.

Websites bevat gegevens over waar Open Zaak gehost wordt. Zorg ervoor dat de standaard website het juiste domein ingesteld heeft (en dus niet example.com).

Logs

Er worden vaak informatieve logberichten weggeschreven die kunnen wijzen op een probleem in de Open Zaak applicatie. Deze worden via de logs inzichtelijk gemaakt.

Failed notifications toont de notificaties die Open Zaak probeerde te versturen, maar om één of andere reden niet slaagden. Je kan hier manueel notificaties opnieuw versturen of verder onderzoeken waarom de notificatie niet kon verstuurd worden.

Logging bevat generieke logberichten die meer informatie kunnen verschaffen over mogelijke foutsituaties.

Lijst- en detailweergaves

De structuur van de admin volgt voor het grootste deel hetzelfde patroon:

1. Vertrek vanaf het dashboard

2. Klik een onderwerp aan binnen een groep, bijvoorbeeld Zaken

3. Vervolgens zie je een lijst van gegevens

4. Na het doorklikken op één item op de lijst zie je een detailweergave

We gaan nu dieper in op wat je kan in lijst- en detailweergaves.

Lijstweergave

Als voorbeeld zie je de lijstweergave van Zaken:

1. De meeste lijstweergaves hebben een zoekveld waarmee je de lijst van gegevens kan doorzoeken. Vaak zoeken deze op identificatie, UUID of een ander karakteristiek attribuut.

2. Aan de rechterzijde is er meestal een set aan filters beschikbaar. Deze laten je toe om snel de resultaatset te reduceren. Filters kunnen gecombineerd worden (combinaties werken als EN-filter). Filteren op startdatum: vandaag en vertrouwelijkheidaanduiding: openbaar toont je alle zaken die vandaag gestart zijn EN de vertrouwelijkheidaanduiding “openbaar” hebben.

3. Kolommen zijn sorteerbaar - klik op het kolomhoofd om oplopend te sorteren. Klik een tweede keer om aflopend te sorteren. Je kan sorteren op meerdere kolommen - er verschijnt dan een nummer die aangeeft op welke kolommen er in welke volgorde gesorteerd wordt.

4. In de lijstweergave zijn bulk acties beschikbaar. Selecteer de objecten waarop je de bulk actie wil toepassen door het vinkje links aan te vinken. Kies vervolgens in de dropdown te actie die je uit wil voeren.

   Warning

   Merk op dat het verwijderen van objecten deze objecten ook echt permanent verwijdert! Het is zelden nodig om objecten te verwijderen.

5. Typisch is de eerste kolom in een lijstweergave een klikbare link. Door deze aan te klikken ga je naar de Detailweergave van dat object.

6. Rechtsboven heb je typisch een knop om nieuwe objecten toe te voegen. Deze opent een formulier om de objectgegevens in te vullen.

Detailweergave

In de detailweergave zie je de gegevens/attributen van één enkel object, al dan niet aangevuld met de gerelateerde objecten.

Als voorbeeld zie je (een deel van) de detailweergave van een zaak:

1. De attributen van de zaak worden opgelijst als bewerkbare velden. Sommige attributen zullen niet bewerkbaar zijn, en als je geen bewerkrechten hebt zie je alles als alleen-lezen. Verplichte velden worden in het vet gedrukt, terwijl optionele velden normaal gedrukt zijn. Indien beschikbaar, dan wordt onder het veld een extra helptekst getoond die meer context geeft over de betekenis van een veld.

2. Gerelateerde objecten worden vaak via een vergrootglas ingesteld. Wanneer je het icoon aanklikt, dan wordt er een lijstscherm geladen waarin je het gerelateerde object kan selecteren. Na selectie staat het database-ID ingevuld in het veld.

3. Gerelateerde objecten worden inline getoond. Zo zul je bij een zaak een aantal inlines zien: statussen, zaakobjecten, zaakinformatieobjecten… Dit zijn allemaal relaties aan deze specifieke zaak.

4. Je kan de geschiedenis inkijken van een specifiek object. Dit toont de wijzigingen aangebracht via de admin interface en door wie en de audit log van wijzigingen die via de API gebeurd zijn.

Wanneer je helemaal naar beneden scrollt (en de juiste rechten hebt), dan zie je links onderin ook een knop Verwijderen. Hierop klikken brengt je naar een bevestigingsscherm. In dit scherm worden alle gerelateerde objecten getoond die mee zullen verwijderd worden.

Warning

Verwijderen van objecten is permanent! Eenmaal je de verwijdering bevestigt kan dit niet meer teruggedraaid worden.

Catalogusbeheer

Note

Om catalogussen te kunnen beheren moet je tot de Catalogi admin groep behoren of equivalente permissies hebben. Zie Groepen voor groepenbeheer.

Catalogus aanmaken

Om aan de gang te kunnen met catalogusbeheer moet er eerst een catalogus aangemaakt worden, klik bij de beginpagina van de admin op Catalogi onder het kopje Gegevens en klik vervolgens op de knop Catalogus toevoegen rechtsbovenin.

Vul op de volgende pagina minimaal alle dikgedrukte velden in en klik daarna op de OPSLAAN knop rechtsonderin de pagina.

Zaaktype aanmaken

Voeg na het toevoegen van de catalogus een Zaaktype toe aan deze catalogus door te klikken op Toon Zaaktypen onder het kopje ACTIES.

Klik vervolgens op de Zaaktype toevoegen knop rechtsbovenin. Op de volgende pagina is het aan te maken zaaktype al gekoppeld aan de juiste catalogus, vul de overige verplichte informatie in en sla het zaaktype op door te klikken op de knop Opslaan en opnieuw bewerken, onderaan de pagina.

Zaaktype publiceren

Het zojuist aangemaakt zaaktype is nog een concept, wat inhoudt dat dit zaaktype niet gebruikt kan worden buiten de Catalogi API zelf (er kunnen bijvoorbeeld nog geen Zaken aangemaakt worden met dit zaaktype). Klik op de Publiceren knop onderaan de detail pagina van het zaaktype om het zaaktype te publiceren.

Note

Als je op de detailpagina van het zaaktype aanpassingen maakt en vervolgens het zaaktype publiceert, dan worden deze aanpassingen ook opgeslagen.

Een nieuwe versie van een zaaktype aanmaken

Om een nieuwe versie van een zaaktype toe te voegen, moet eerst de datum einde geldigheid van het zaaktype ingevuld worden.

Klik zodra deze datum ingevuld is op de Nieuwe versie toevoegen knop om een nieuwe versie van het zaaktype aan te maken.

Navigeer vervolgens naar de zaaktypen van de aangemaakte catalogus om de nieuwe versie te zien. De nieuwe versie zal eerst gepubliceerd moeten worden voor gebruik buiten de Catalogi API.

Exporteren/importeren van een catalogus

Een catalogus kan samen met alle typen die erin zitten (Zaaktypen, Informatieobjecttype, etc.) geëxporteerd worden naar een .zip archief, dat vervolgens weer gebruikt kan worden om de catalogus in een andere Catalogi API te importeren.

Om dit te doen in OpenZaak, klik op de te exporteren catalogus onder Gegevens > Catalogi en klik vervolgens op de Exporteren knop onderaan de pagina. Download daarna de export als .zip-bestand.

Om de importfunctionaliteit te demonstreren is de zojuist geëxporteerde catalogus verwijderd uit de OpenZaak admin. Klik hiervoor op de Verwijderen knop linksonderin de detailpagina van de catalogus.

Importeer de catalogus door op de catalogus lijstweergave te klikken op de Importeer catalogus knop rechtsbovenin. Upload op de volgende pagine het .zip-bestand en kies of er voor de objecten nieuwe UUIDs gegenereerd moeten worden, of dat de bestaande UUIDs uit de import gebruikt moeten worden.

Warning

LET OP: alle Zaaktypen, Informatieobjecttypen en Besluittypen worden geïmporteerd als concept.

Exporteren/importeren van een zaaktype

In sommige gevallen hoeft niet een gehele catalogus geïmporteerd te worden, maar alleen een enkel zaaktype uit die catalogus, dit is ook mogelijk in de OpenZaak admin.

Om te demonstreren hoe het importeren werkt als er Informatieobjecttypen en Besluittypen gerelateerd zijn aan het Zaaktype, worden deze voor deze tutorial eerst toegevoegd aan het zaaktype. Navigeer hiervoor naar de catalogi lijstweergave, klik op Toon alle besluittypen, klik daarna op Besluittype toevoegen, vul de verplichte informatie in en leg een relatie met het te exporteren zaaktype (hetzelfde geldt voor Informatieobjecttypen).

Exporteer het zaaktype door te klikken op Export onderaan de pagina van het zaaktype en download het .zip-bestand. Om hierna het importeren te demonstreren, wordt dit zaaktype verwijderd door te klikken op Verwijderen linksonderin de zaaktype pagina.

Klik vervolgens de catalogus aan waarin het zaaktype terecht moet komen en druk op de Import ZaakType knop.

Upload op de volgende pagina het .zip-bestand met de export van het zaaktype

Omdat er in de .zip ook Besluittypen en Informatieobjecttypen zitten, moet er bepaald worden of deze ook geïmporteerd moeten worden, of dat deze vervangen kunnen worden door bestaande Besluittypen en Informatieobjecttypen. Aangezien het Besluittype en Informatieobjecttype in deze tutorial niet zijn verwijderd, wordt er hier voor gekozen om de bestaande typen te gebruiken. Klik tot slot op Select om de import uit te voeren.

Gebruikerstoegang

Note

Om gebruikers te kunnen beheren moet je tot de User admin groep behoren of equivalente permissies hebben. Zie Groepen voor groepenbeheer.

Groepen

Een groep is een set van permissies. Een permissie laat een gebruiker toe om iets te doen. Typisch zijn er vier soorten permissies voor een soort van gegeven:

• objecten lezen

• objecten aanmaken

• objecten aanpassen

• objecten verwijderen

Een gebruiker kan tot meerdere groepen behoren - de permissies vullen elkaar dan aan, d.w.z. dat je de gecombineerde set van permissies krijgt van elke groep.

Een standaard Open Zaak installatie komt met een aantal standaardgroepen:

Admin

Een beheerder die alles kan. Ga hier erg zorgvuldig mee om!

API admin

Leden van deze groep kunnen taakapplicaties en hun API-toegang instellen, en Open Zaak configureren om andere API’s te consumeren.

Autorisaties admin

Leden van deze groep kunnen taakapplicaties en hun API-toegang instellen, en Open Zaak configureren om andere API’s te consumeren.

Autorisaties lezen

Leden van deze groep kunnen zien welke applicaties toegang hebben tot de Open Zaak API’s, maar deze niet bewerken.

Besluiten admin

Leden van de groep kunnen de resources ontsloten via de Besluiten API lezen, aanmaken en bewerken:

• Besluiten

• Relaties tussen besluiten en documenten

Besluiten lezen

Leden van de groep kunnen de resources ontsloten via de Besluiten API lezen.

Catalogi admin

Leden van de groep kunnen de resources ontsloten via de Catalogi API lezen, aanmaken en bewerken:

• Catalogi

• Zaaktypen

• Statustypen

• Resultaattypen

• Eigenschappen

• Roltypen

• Informatieobjecttypen

• Besluittypen

• Zaaktype-informatieobjecttypen

Catalogi lezen

Leden van de groep kunnen de resources ontsloten via de Catalogi API lezen.

Documenten admin

Leden van de groep kunnen de resources ontsloten via de Documenten API lezen, aanmaken en bewerken:

• Enkelvoudige informatieobjecten (= documenten) met versiehistorie

• Gebruiksrechten van documenten

• Relaties tussen documenten en andere objecten

Documenten lezen

Leden van de groep kunnen de resources ontsloten via de Documenten API lezen.

Zaken admin

Leden van de groep kunnen de resources ontsloten via de Zaken API lezen, aanmaken en bewerken:

• Zaken

• Statussen van zaken

• Resultaat van zaken

• Eigenschappen van zaken

• Documenten bij zaken

• Andere objecten bij zaken

• Klantcontacten van zaken

• Betrokkenen bij zaken

• Relevante zaak-relaties

Zaken lezen

Leden van de groep kunnen de resources ontsloten via de Zaken API lezen.

Een gebruiker aanmaken (en aan een groep toevoegen)

Vertrek vanaf de startpagina en vind de groep Accounts. Klik vervolgens op Toevoegen naast de titel Gebruikers:

Vul vervolgens een gebruikersnaam in en een wachtwoord. Vergeet niet om het wachtwoord te bevestigen. Merk op dat er wachtwoordsterkte-regels gelden!

Note

Gebruikersnamen zijn hoofdlettergevoelig!

Klik vervolgens op Opslaan en opnieuw bewerken:

In het volgende scherm kan je vervolgens de rechten voor deze gebruiker instellen.

1. Het vinkje Stafstatus bepaalt of de gebruiker in kan loggen op de admin-omgeving.

2. Het vinkje Supergebruikerstatus geeft aan of de gebruiker altijd alle permissies heeft. We raden sterk aan om hier conservatief mee om te gaan.

3. De beschikbare groepen staan opgelijst in de linkerkolom. Om een gebruiker aan een groep toe te wijzen selecteer je de groep door deze aan te klikken en op de pijl naar rechts (4) te klikken. Je kan meerdere groepen in één keer toekennen door de CTRL-toets ingedrukt te houden tijdens het aanklikken. Je kan ook dubbelklikken om een groep toe te kennen.

Klik rechtsonder op Opslaan om de wijzigingen door te voeren.

API-toegang beheer

Open Zaak biedt API’s aan voor zaakgericht werken. Deze API’s zijn niet toegankelijk zonder authenticatie en autorisatie - gegevens worden namelijk correct beveiligd. Eindgebruikers maken gebruik van (taak)applicaties, en het zijn deze applicaties die gebruik maken van de API’s voor zaakgericht werken.

Voordat een applicatie dus gegevens kan opvragen, opvoeren, bewerken of vernietigen, moet deze applicatie hiervoor bekend zijn en geautoriseerd zijn.

Elke applicatie die aangesloten wordt op Open Zaak krijgt authenticatiegegevens. Tegelijkertijd worden autorisaties ingesteld per applicatie. Autorisatiebeheerders dienen vervolgens de applicatie-authenticatiegegevens te communiceren naar de beheerder(s) van de (taak)applicatie.

Warning

De applicaties/authenticatiegegevens die hier beheerd worden, zijn niet geschikt voor eindgebruikers, maar enkel voor server-to-server communicatie.

Note

Om API-toegang te kunnen beheren moet je tot de API admin groep behoren of equivalente permissies hebben. Zie Groepen voor groepenbeheer.

Een Applicatie registreren

Klik op het dashboard onder de groep API Autorisaties op de link Toevoegen naast de kop Applicaties:

Vul vervolgens het formulier in:

Het label is een vriendelijke naam voor de applicatie waarmee je kan herkennen om welke applicatie binnen je organisatie het gaat. Je kan dit vrij kiezen.

Het vinkje Heeft alle autorisaties laat je toe om snel een applicatie alle mogelijke rechten te geven. Merk op dat dit bijna altijd betekent dat een applicatie meer mag dan nodig is!

Zorg ervoor dat je minstens één combinatie van Client ID en Secret genereert. De applicatie heeft deze gegevens nodig om tokens te kunnen genereren - communiceer deze dus op een veilige manier naar de beheerder van de (taak)applicatie.

Klik tot slot op Opslaan en opnieuw bewerken, waarna je de autorisaties in kan stellen.

Instellen van de API-toegang voor een Applicatie

Je kan de applicatie waarvan je autorisaties wenst in te stellen bereiken via het lijstoverzicht of vlak na het registreren van een nieuwe applicatie.

Klik rechtsonderin op Beheer autorisaties om de autorisaties van een applicatie in te stellen.

Autorisaties zijn van toepassing op een bepaalde API-component. Eén applicatie kan zonder problemen autorisaties hebben op een set van componenten.

Componentkeuze

Kies in de eerste stap voor welke component je autorisaties wil instellen:

Afhankelijk van de keuze van de component gebeuren nu een aantal dingen:

• de relevante scopes worden nu getoond

• eventuele extra keuzeparameters zijn beschikbaar

Keuze scopes

Scopes bepalen welke acties een applicatie via de API kan uitvoeren. Het is een groepering van rechten.

De betekenis van de scopes is gedocumenteerd per component:

• Zaken API

• Documenten API

• Catalogi API

• Besluiten API

• Notificaties API

• Autorisaties API

Typisch zal een leverancier van de applicatie je vertellen welke scopes er precies nodig zijn. Selecteer de relevante scopes door de checkboxes aan te vinken.

Keuze relevante typen

Voor de Zaken API, Documenten API en Besluiten API worden de autorisaties ingericht per type - Zaaktype, Informatieobjecttype en Besluittype respectievelijk.

De type-selectie heeft drie mogelijke opties:

Alle huidige *typen:

Dit selecteert alle huidige typen, ook de typen die nog niet gepubliceerd zijn. Voor de Zaken API gelden de autorisaties dan voor alle zaaktypen uit alle catalogi in Open Zaak.

Alle huidige en toekomstige *typen:

Dit is dezelfde situatie als Alle huidige *typen, met het verschil dat *typen die aangemaakt worden na het instellen van de autorisaties hier ook binnen vallen.

Selecteer handmatig:

Bij handmatige selectie worden alle *typen per catalogus opgelijst. Kies de relevante *typen aan door het vinkje aan te zetten.

Keuze vertrouwelijkheidaanduiding

Tot slot kan je limiteren tot welke vertrouwelijkheidaanduiding een applicatie toegang heeft. Documenten en Zaken worden aangemaakt met een bepaalde vertrouwelijkheidaanduiding. Een applicatie heeft enkel toegang tot deze documenten en zaken waarvan de vertrouwelijkheidaanduiding meer publiek of gelijk-aan de ingestelde autorisatie-vertrouwelijkheidaanduiding is.

Voor de applicatie lijkt het alsof Documenten en Zaken die meer vertrouwelijk zijn niet bestaan.

Note

Merk op dat de ingestelde vertrouwelijkheidaanduiding van toepassing is op de geselecteerde zaaktypen of informatieobjecttypen. Je kan dus een verschillende maximale vertrouwelijkheidaanduiding instellen per zaaktype of informatieobjecttype.

Toevoegen extra autorisaties

Linksonderin vind je een link Nog Autorisaties toevoegen. Zodra je hierop klikt, ga je opnieuw door hetzelfde proces om een autorisatie in te stellen. Je kan zoveel autorisaties instellen als je wenst.

Warning

Let op dat je geen conflicterende autorisaties instelt. Eenmaal een autorisatie voor (een groep van) zaaktypen, informatieobjecttypen of besluittypen is ingesteld mag je geen extra autorisaties voor diezelfde (groep van) zaaktypen, informatieobjecttypen of besluittypen meer instellen.

Opslaan wijzigingen

Wanneer je klaar bent, vergeet dan niet om rechtsonderin op Opslaan te klikken om de autorisaties te bewaren.

Gebruik van externe API’s

Open Zaak past de Common Ground gedachte toe van data-bij-de-bron ophalen. Er bestaan al een aantal RESTful API’s die gebruikt kunnen worden met Open Zaak, zoals de BAG en BRT. Een praktisch voorbeeld hiervan is het relateren van een Pand of Wegdeel aan een zaak.

Deze API’s hebben echter vaak hun eigen authenticatie-mechanisme, zelfs als het Open Data betreft - vaak ter voorkoming van misbruik. Er zijn twee mogelijke pistes om deze APIs te kunnen gebruiken met Open Zaak:

• Instellen van Authenticatie voor deze API’s

• API’s consumeren via NLX

Deze kunnen ook gecombineerd worden - consumeren via NLX én gebruik van een API key.

Authenticatie

De Kadaster en HaalCentraal API’s vereisen typisch authenticatie met een API key. Consulteer de documentatie van de betreffende API om een API key te verkrijgen.

Navigeer daarna in de admin interface naar API Autorisaties en klik op Other external API credentials.

Klik rechtsboven op Other external API credential toevoegen en vul de velden in:

API-root

de basis-URL waar de API beschikbaar is, bijvoorbeeld http://bag.basisregistraties.overheid.nl/api/v1

Label

Een omschrijving zodat je herkent om welke API het gaat, bijvoorbeeld “BAG”.

Header key

De naam van de HTTP header waar de API key in gaat, bijvoorbeeld X-Api-Key of Authorization.

Header value

De waarde van de API key die je eerder verkreeg.

Klik tot slot op Opslaan.

Note

Bij het aanmaken van bijvoorbeeld ZaakObjecten wordt er gevalideerd dat de URL van het object geldig is. Om dit te kunnen doen, moet Open Zaak weten hoe zich te autoriseren voor deze URL. De bovenstaande procedure maakt dit mogelijk.

NLX

NLX faciliteert data-uitwisseling voor overheden en organisaties. Open Zaak heeft functionaliteiten die het gebruik van NLX aanmoedigen.

Organisaties kunnen ervoor kiezen om de gegevens via NLX te ontsluiten in een opzet waarbij geen API key nodig is voor de consumer, zolang deze maar via het NLX netwerk de gegevens opvraagt.

Open Zaak kan deze “publieke” URLs vertalen naar NLX-specifieke URLs zodat de gegevens via het NLX netwerk opgevraagd worden. Hiervoor moet je een NLX outway beschikbaar hebben.

Navigeer in de admin interface naar Configuratie > URL rewrites. Klik op URL Rewrite toevoegen en vul het formulier in:

From value:

Het beginstuk van de URL die je wenst om te zetten. Bijvoorbeeld http://bag.basisregistraties.overheid.nl/api/v1

To value:

De equivalente URL in je outway, bijvoorbeeld http://my-outway:8443/kadaster/bag.

Note

Consulteer de NLX directory om te zien welke services beschikbaar zijn.

ADFS (On premise) Configureren

Open Zaak ondersteunt Single Sign On (SSO) via ADFS voor de beheerinterface.

Gebruikers kunnen op die manier inloggen op Open Zaak met hun ADFS account. In deze flow:

1. Klikt een gebruiker op het inlogscherm op Inloggen met ADFS

2. De gebruiker wordt naar de ADFS omgeving geleid waar ze inloggen met gebruikersnaam en wachtwoord (en eventuele Multi Factor Authorization)

3. De ADFS omgeving stuurt de gebruiker terug naar Open Zaak (waar de account aangemaakt wordt indien die nog niet bestaat)

4. Een beheerder in Open Zaak kent de juiste groepen toe aan deze gebruiker als deze voor het eerst inlogt.

Note

Standaard krijgen deze gebruikers geen toegang tot de beheerinterface. Deze rechten moeten door een (andere) beheerder ingesteld worden. De account is wel aangemaakt.

Configureren van ADFS zelf

Contacteer de IAM/ADFS-beheerders in je organisatie om een Application Group aan te maken in de ADFS omgeving. Er zijn configuratiehandleidingen (Engels) beschikbaar.

Voor de Redirect URI vul je https://open-zaak.gemeente.nl/adfs/callback in, waarbij je open-zaak.gemeente.nl vervangt door het relevante domein.

Aan het eind van dit proces moet je de volgende gegevens hebben (on premise):

• Server adres, bijvoorbeeld login.gemeente.nl

• Client ID, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6

Configureren van ADFS in Open Zaak

Zorg dat je de volgende gegevens hebt:

• Server adres

• Client ID

Navigeer vervolgens in de admin naar Configuratie > ADFS Confiugration.

1. Vink Enable aan om ADFS in te schakelen.

2. Vul bij Server (on premise) het server adres in, bijvoorbeeld login.gemeente.nl.

3. Vul bij Client ID het Client ID in, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6.

4. Vul bij Relying Party ID opnieuw het Client ID in, bijvoorbeeld 3ae1852d-bf76-4731-9c41-1a31928cf6a6.

5. Laat bij Claim mapping de standaardwaarden staan.

6. Vul bij Username claim de waarde winaccountname in.

Klik tot slot rechtsonder op Opslaan.

Je kan vervolgens het makkelijkst testen of alles werkt door in een incognitoscherm naar https://open-zaak.gemeente.nl/admin/ te navigeren en op Inloggen met ADFS te klikken.

Open Zaak client documentation

Open Zaak is primarily a provider of API’s to be consumed by clients. If you’re developing such a client (or consumer), you’re in the right place!

Please select your relevant topic

Authentication and authorization

All endpoints are authorization protected, per the upstream Standard.

Open Zaak uses the described authentication and authorization mechanism based on JSON Web Tokens (JTW in short). You can read more about JWT’s on https://jwt.io

To connect to Open Zaak, you have received a Client ID and a Secret, which you must use to build a JWT.

The payload of the JWT is:

{
    "iss": "<value of Client ID>",
    "iat": 1602857301,
    "client_id": "<value of Client ID>",
    "user_id": "<unique user ID of the actual end user>",
    "user_representation": "<e.g. the name of the actual end user>"
}

The JWT must be generated with the HS256 algorithm and signed with the secret you received. iat is the Unix timestamp when token-creation happened.

Next, for every API call you make, you must include this token in the appropriate header:

Authorization: Bearer <jwt>

An example:

Given a Client ID docs and secret example-secret, the header of the JWT is:

{
    "typ": "JWT",
    "alg": "HS256"
}

The payload is:

{
  "iss": "docs",
  "iat": 1602857301,
  "client_id": "docs",
  "user_id": "docs@example.com",
  "user_representation": "Documentation Example"
}

Which leads to the following JWT:

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJkb2NzIiwiaWF0IjoxNjAyODU3MzAxLCJjbGllbnRfaWQiOiJkb2NzIiwidXNlcl9pZCI6ImRvY3NAZXhhbXBsZS5jb20iLCJ1c2VyX3JlcHJlc2VudGF0aW9uIjoiRG9jdW1lbnRhdGlvbiBFeGFtcGxlIn0.DZu7E780xG4zqRiT8ZhrBeMudz45301wNVDT0ra-Iyw

This would then be used in an API call like:

GET https://test.openzaak.nl/besluiten/api/v1/besuiten HTTP/1.1
Host: test.open-zaak.nl
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJkb2NzIiwiaWF0IjoxNjAyODU3MzAxLCJjbGllbnRfaWQiOiJkb2NzIiwidXNlcl9pZCI6ImRvY3NAZXhhbXBsZS5jb20iLCJ1c2VyX3JlcHJlc2VudGF0aW9uIjoiRG9jdW1lbnRhdGlvbiBFeGFtcGxlIn0.DZu7E780xG4zqRiT8ZhrBeMudz45301wNVDT0ra-Iyw

Warning

Note that you are expected to generate a JWT almost for every call! Open Zaak by default expires JWT’s one hour past the iat timestamp, and for audit-purposes, the user_id and user_representation claims should match the end-user of the application.

Cross-Origin Resource Sharing (CORS)

Some clients develop against Open Zaak using single-page-application technology that runs completely in the browser, such as React, Angular or other frameworks.

Open Zaak must be deployed with an appropriate CORS-configuration for this.

Note

We always recommend using an API gateway/own backend to communicate with Open Zaak. It’s simpler because you don’t have to deal with CORS, and there’s less risk of credentials/secrets leaking. You should never store client ID/secret in your dist bundle(s).

Production-grade settings

In production-like environments, we recommend using an explicit allow-list for the trusted origins. This requires deploying Open Zaak with CORS_ALLOWED_ORIGINS=https://my-app.example.com, where https://my-app.example.com is the domain where the application is deployed.

Development/experimental configuration

If you’re running Open Zaak locally or on an environment with dummy data for development purposes, you can grant CORS access to every possible client using CORS_ALLOW_ALL_ORIGINS=True in the Open Zaak deployment.

Separation of administrative interface and API

The administrative interface authenticates using session cookies, while the APIs use the Authorization header with bearer tokens.

The session cookies are never sent on cross-domain requests, and the CORS configuration is configured to not allow credentials (which are typically session cookies). The API with the Authorization header is not affected by this policy.

Recipes

In the recipes documentation, we aim to describe some patterns to organize your API calls to maximize performance.

If you can give the equivalent example in your own language-of-preference, please submit a pull request!

Creating a Zaak, a Document and relating them

Python (Django)

Javascript

Using Django with zgw-consumers

import base64
from datetime import date

from zgw_consumers.constants import APITypes
from zgw_consumers.models import Service

zrc_client = Service.objects.filter(api_type=APITypes.zrc).get().build_client()
drc_client = Service.objects.filter(api_type=APITypes.drc).get().build_client()

# zaak creation
today = date.today().strftime("%Y-%m-%d")
zaak_body = {
    "zaaktype": "https://test.openzaak.nl/catalogi/api/v1/zaaktypen/4acb5ab8-f189-4559-b18a-8a54553a74ff",
    "bronorganisatie": "123456782",
    "verantwoordelijkeOrganisatie": "123456782",
    "registratiedatum": today,
    "startdatum": today,
}
zaak: dict = zrc_client.create("zaak", zaak_body)

# document creation
with open("/tmp/some_file.txt", "rb") as some_file:
    document_body = {
        "bronorganisatie": "123456782",
        "creatiedatum": today,
        "titel": "Example document",
        "auteur": "Open Zaak",
        "inhoud": base64.b64encode(some_file.read()).decode("utf-8"),
        "bestandsomvang": some_file.size,
        "bestandsnaam": some_file.name,
        "taal": "nld",
        "informatieobjecttype": (
            "https://test.openzaak.nl/catalogi/api/v1/"
            "informatieobjecttypen/abb89dae-238e-4e6a-aacd-0ba9724350a9"
        )
    }
document: dict = drc_client.create("enkelvoudiginformatieobject", document_body)

# relate them
zio_body = {
    "zaak": zaak["url"],
    "informatieobject": document["url"],
}
zio: dict = zrc_client.create("zaakinformatieobject", zio_body)

import jwt from 'jsonwebtoken';

const CLIENT_ID = 'example';
const SECRET = 'secret';

// helpers
const getJWT = () => {
  return jwt.sign(
    {client_id: CLIENT_ID},
    SECRET,
    {
      algorithm: 'HS256',
      issuer: CLIENT_ID,
    }
  );
};

const apiCall = (url, method='get', body) => {
  const fetchBody = body ? JSON.stringify(body) : null;
  const token = getJWT();
  const response = await fetch(
    url,
    {
      method: method,
      headers: {
        'Authorization': `Bearer ${token}`,
        'Accept': 'application/json',
      },
      body: _body
    }
  );
  const responseData = await response.json();
  return responseData;
};

const toBase64 = file => new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(file);
    reader.onload = () => resolve(reader.result);
    reader.onerror = error => reject(error);
});

// zaak creation
const today = '2020-10-16';
const zaakBody = {
    'zaaktype': 'https://test.openzaak.nl/catalogi/api/v1/zaaktypen/4acb5ab8-f189-4559-b18a-8a54553a74ff',
    'bronorganisatie': '123456782',
    'verantwoordelijkeOrganisatie': '123456782',
    'registratiedatum': today,
    'startdatum': today,
}
const zaak = await apiCall(
  'https://test.openzaak.nl/zaken/api/v1/zaken',
  'POST',
  zaakBody
);

// document creation
const someFile = document.querySelector('#myfile').files[0];
const documentBody = {
  'bronorganisatie': '123456782',
  'creatiedatum': today,
  'titel': 'Example document',
  'auteur': 'Open Zaak',
  'inhoud': toBase64(someFile),
  'bestandsomvang': someFile.size,
  'bestandsnaam': someFile.name,
  'taal': 'nld',
  'informatieobjecttype': `https://test.openzaak.nl/catalogi/api/v1/
informatieobjecttypen/abb89dae-238e-4e6a-aacd-0ba9724350a9`
};

const doc = await apiCall(
  'https://test.openzaak.nl/documenten/api/v1/enkelvoudiginformatieobjecten',
  'POST',
  documentBody
);

// relate them
const zioBody = {
  'zaak': zaak.url,
  'informatieobject': doc.url
};
const zio = await apiCall(
  'https://test.openzaak.nl/zaken/api/v1/zaakinformatieobjecten',
  'POST',
  zioBody,
);

Retrieving the documents related to a Zaak

Key problem here is that one Zaak has (probably) multiple documents related to it, and retrieving them sequentially is a performance hit that gets worse with the amount of documents.

The solution is to use some form of threading/concurrency offered by your language.

Python (Django)

Javascript

Using Django with zgw-consumers, we can use the concurrent.fututures.ThreadPoolExecutor, where each job will run in its own thread. This gets close to retrieving all the documents in parallel instead of sequentially, resulting in a constant-time determined by the slowest API call.

from typing import List

from zgw_consumers.constants import APITypes
from zgw_consumers.models import Service
from zgw_consumers.concurrent import parallel

zrc_client = Service.objects.filter(api_type=APITypes.zrc).get().build_client()
drc_client = Service.objects.filter(api_type=APITypes.drc).get().build_client()

zaak_url = "https://test.openzaak.nl/zaken/api/v1/zaken/b604ea56-f01c-432e-8d61-fd4ab02893dc"
zios: List[dict] = zrc_client.list("zaakinformatieobject", {"zaak": zaak_url})
document_urls = [zio["informatieobject"] for zio in zios]
with parallel() as executor:
    _documents = executor.map(
        lambda url: drc_client.retrieve("enkelvoudiginformatieobject", url=url),
        document_urls
    )
documents: List[dict] = list(_documents)

Similarly to the Python case, we leverage the Javacsript async/await event loop. Once we’ve collected all the URLs of documents to retrieve, we create promises and by using Promise.all, all API calls are being performed in parallel (at least for the network IO part).

import jwt from 'jsonwebtoken';

const CLIENT_ID = 'example';
const SECRET = 'secret';

// helpers
const getJWT = () => {
  return jwt.sign(
    {client_id: CLIENT_ID},
    SECRET,
    {
      algorithm: 'HS256',
      issuer: CLIENT_ID,
    }
  );
};

const get = (url) => {
  const token = getJWT();
  const response = await fetch(
    url,
    {
      method: 'get',
      headers: {
        'Authorization': `Bearer ${token}`,
        'Accept': 'application/json',
      },
    }
  );
  const responseData = await response.json();
  return responseData;
};

const zaakUrl = 'https://test.openzaak.nl/zaken/api/v1/zaken/b604ea56-f01c-432e-8d61-fd4ab02893dc';
const zios = await get(`https://test.openzaak.nl/zaken/api/v1/zaakinformatieobjecten?zaak=${zaakUrl}`);
const promises = zios.map(zio => get(zio.informatieobject));
const documents = await Promise.all(promises);

Support

Development is done separately from any support or service that you might need to install, use, implement or integrate Open Zaak.

What kind of support do you need?

• Want to get Open Zaak? Contact a service provider!

• Need help? Contact the maintainer of Open Zaak or a service provider!

• Found a bug? Please create an issue on Github!

• Found a security issue? Please contact the maintainer of Open Zaak directly!

• Want to contribute? See Development!

Community support

The maintainer, other users, developers and service providers can be found on these channels:

• Open Zaak channel on the Samen Organiseren Slack (join here)

• Open Zaak group on the Common Ground website

Service providers

Service providers offer Open Zaak as a service or can manage an on-premise installation for you.

Below is a list of service providers.

• Maykin Media (maintainer of Open Zaak)

• Contezza

More to come!

Maintainer

The maintainer is responsible for bug fixes, security fixes, code reviews and new releases.

Reporting security issues

Open Zaak’s security policies

Open Zaak’s development team is strongly committed to responsible reporting and disclosure of security-related issues. As such, we’ve adopted and follow a set of policies which conform to that ideal and are geared toward allowing us to deliver timely security updates to the official distribution of Open Zaak.

Reporting security issues

Short version: please report security issues by emailing security@maykinmedia.nl.

If you discover security issues in Open Zaak or related projects under the same organization, we request you to disclose these in a responsible way by e-mailing to security@maykinmedia.nl.

It is extremely useful if you have a reproducible test case and/or clear steps on how to reproduce the vulnerability.

Please do not report security issues on the public Github issue tracker, as this makes it visible which exploits exist before a fix is available, potentially comprising a lot of unprotected instances.

Once you’ve submitted an issue via email, you should receive an acknowledgment from a member of the security team as soon as possible, and depending on the action to be taken, you may receive further followup emails.

Timeline of the process

Open Zaak has a technical steering group, of which all members are involved in the handling of security issues.

1. The recipients of the report first validate if there is indeed a (possible) issue.

2. After validation, we confirm that we received the report and if it is indeed a valid issue.

3. We have a private Github repository accessible to the technical steering group. In this repository, an issue is created for the vulnerability where the impact and possible solutions are discussed.

4. The next step is to create a (draft) Github security advisory, which is only visible to the repository administrators and technical steering group. Severity and impact will be established here.

5. If appropriate, we request a CVE identifier from Github.

6. A patch is implemented, reviewed and tested in a private fork.

7. During the patch development process, known service providers are contacted to inform them of the vulnerability and coordinate the release date and rollout of the fix. Service providers should subscribe to the release early notice list.

8. When the fix is tested and release coordination is done, the fix is merged into the primary repository. The security advisory and release are published. Service providers update their managed instances.

9. The release and security vulnerability are communicated to the community. This includes a message to the mailing list and announcements on the Common Ground Slack and on commonground.nl.

Development

Open Zaak is Open-source software. We’d love to have you contribute!

Roadmap

The roadmap for Open Zaak includes high-level overview of future and in-progress development efforts. These can be feature-oriented, or aimed at cleaning up technical debt in the project and/or enhancing the meta-stuff around the actual development.

Bugfixing

Bugs are considered defects. Bugs must be classified by severity:

• data loss risk

• security vulnerability

• annoying but workaround available

The first two are high-prio and should be fixed as soon as possible, the third item is low prio.

Features

Feature implementation is the connection between the product steering team and the technical team. The product steering team will usually bring up request features or consider the requested (big) features from the Github issues.

Current list is created from issue skimming on Github.

Post-installation configuration

After installation, some configuration must now be done in the admin interface. However, a user-friendly initial-setup wizard could be provided that simplifies this and collects the fragmented aspects in an intuitive user interface.

Impact

• would drastically simplify post-installation initial configuration

• affects new installations by service providers installing Open Zaak

• sizing: medium

i-Navigator (and others) support

There are existing applications/tools containing catalogus data. Supporting these data formats is beneficial to adoption of Open Zaak since the users don’t have to type over all the data. However, it’s not necessary an Open Zaak thing because it should make use of the API standard.

Impact

• improves adoption

• affects people maintaining/migrating zaaktype-catalogi to Open Zaak

• not sure if this is our scope

• sizing: large

Better Catalogus management interface

Currently the content is managed through the Django admin interface with some customization, which is limited in how far you can beat it into submission.

There should be a separately developed project to interface, based on the API standard which provides a more native (=well integrated with the tree structure) interface to manage catalogi, zaaktypen, informatieobjecttypen, besluittypen and any related data.

Impact

• affects people maintaining catalogi (managing zaaktypes etc.)

• would probably be it’s own, independent project that could be installed along Open Zaak, leaning on OZ’s user management

• sizing: large

Improve archiving parameters integration

Choices you make with an effect on archiving parameters are now validated server side, which is an annoying feedback cycle for administrators. The interface can include the logic to automatically show/hide/filter the available dependent options.

Impact

• affects people maintaining catalogi (managing zaaktypes etc.)

• once done, reduces frustration and decreases amount of mistakes

• sizing: small/medium

Implement password reset functionality

This is currently not available, which requires admin-user intervention if a user forgets their password.

Impact

• relieves administrators of Open Zaak to reset passwords

• prevents administrators knowing Open Zaak users’ passwords

• sizing: small

Provide selectielijst.openzaak.nl on NLX

Related to https://github.com/open-zaak/open-zaak/issues/644 - ensure that firewalls can stay closed by giving the option to use NLX.

Impact

• increases inter-operability

• affects system/networks administrators

• sizing: small

Documentation

Provide documentation for the Ansible collection

The Ansible collection is really the heart of the deployment infrastructure/code and currently a magical black box. The collection itself should be documented, with target audiences: developers, devops, sysadmins.

Impact

• affects possible contributors

• may provide a reference for service providers on how to correctly install OZ

• sizing: medium

User manual: add selectielijst configuration

The user-interface for the configuration of selectielijsten must be included in the manual for day-to-day administrators managing the content of catalogi.

Impact

• affects Open Zaak administrators / people maintaining catalogi

• sizing: small

Technical debt / clean-up / codebase quality

Inclusion of API specs for validation

Currently, OAS specs are linked to (raw.github.com) for validation of remote resources. Instead of requiring an open internet connection to github, we should fetch these specs at build time and include them as static files in the Docker image. This is good for performance, security and reliability.

See https://github.com/open-zaak/open-zaak/issues/644

Impact

• affects system/network administrators

• allows firewalls to (stay or) be more strict

• small performance improvement

• sizing: small

Check if we can change the API timezone to UTC and interface TZ to Europe/Amsterdam

This would display the correct local times for users browsing in the admin interface, while keeping API times in UTC for simplicity.

Impact

• affects people maintaining catalogi in the admin

• affects people investigating data in the admin (zaken, documenten…)

• sizing: small

Setup requires.io integration

This is a (free) service to monitor dependencies that are either out-of-date or have security vulnerabilities. Github handles the security vulnerabilities well, but you want to quickly see if you can update other deps without breaking changes, so you don’t lag behind making the upgrade harder.

Impact

• affects Open Zaak development

• improves Open Zaak security

• sizing: small

Refactor ``FooConfig`` to use ``zgw_consumers.Service``

In various places, we configure API root URLs for which service to use ( notifications API, authorizations API…). Additionally, we also must configure auth/NLX through the services for these endpoints. It would make more sense to centralize the service config and point to a particular service instead of storing the API root again.

Impact

• affects new installation configuration

• affects complexity of codebase (makes it less complex/confusing)

• sizing: small

Include newer Postgres versions in CI

Currently Open Zaak is tested against Postgres 10, while 11 and 12 are out. A test matrix for all versions of Open Zaak seems appropriate.

Impact

• demonstrates compatibility

• explicit support gives more possible target deploy platforms

• sizing: small

Prepare update to Django 3.x

Recently Django 3.1 was released, after 3.0. Open Zaak is on Django 2.2 (LTS). We plan to jump from LTS to LTS - Django 3.2 (LTS) should be released around April 2021.

Impact

• security & future security

• affects: developers, contributors, users (API clients), municipalities with a deployed version

• sizing: medium

Structurally check security updates

Django publishes patch releases at the beginning of each month. Open Zaak should include those as soon as possible for security and stability reasons. We can also check at the same time if other dependencies can/should be updated to new patch releases.

Impact

• good security record

• sizing: small

Developer tooling/experience

Tick of FFPC items

The checklist from the Foundation For Public Code includes a number of project-setup improvements that could/should help get potential contributors started.

Impact

• ability to say you’re FFPC compliant :-)

• sizing: small

Document dev/virtualenv setup

There are some best-practices w/r to storing KUBECONFIG in project-specific locations and/or installing the Ansible dependencies inside of the virtualenv instead of the global system directories. This should be documented with an example setup.

Impact

• less confusion for (potential) service providers by having a reference

• sizing: small

Automate the Ansible collection publishing to Ansible galaxy

Currently, publishing is a manual action by uploading the artifact through the browser.

This can be automated after a succesful CI build on Travis instead, which would also make it easier for committers other than Joeri/Sergei to publish changes.

Impact

• removes manual step from contributors

• removes need to manage auth/permissions on Ansible Galaxy

• sizing: small

Docker Hub paid plan

Open Zaak & related Docker images are published on Docker Hub, which is a free and public image registry. Recently Docker Hub announced changes to the image retention policy for free plans, which will have an impact for organizations running on older versions that are not frequently pulled/updated.

To guarantee availability, alternative solutions should be researched or consider signing up the Open Zaak organization to a paid plan.

Impact

• research with likely financial implications

• not doing it might break deployed (older) Open Zaak versions, in particular patch releases

• sizing: medium

Add automated OAS-comparison to the standard

We should have a (cron) job on the CI to check that the (semantics of the) API specs are still the same as the upstream standard API specs.

Order/encoding does not matter, so we should compare the resolved python dicts/objects.

Impact

• automation of staying compliant with the upstream standard

• sizing: medium

Changelog

1.4.0 (2021-04-30)

New features

• Updated ADFS-integration support, now Azure AD is properly supported

• Allow selection of internal zaaktypen for related zaaktypen with user friendly picker (#910)

• Removed the need to register internal services as external services when using CMIS adapter (#938)

• More CMIS-adapter optimization

  • caching of WSDLs

  • use connection pooling for CMIS requests (#956)

• Added support for initial superuser creation via environment variables (#952)

Bugfixes

• Updated to Zaken API 1.0.3 specification, see the upstream 1.0.3 changelog.

  • rol_list operation querystring parameter fixed, from betrokkeneIdentificatie__vestiging__identificatie to betrokkeneIdentificatie__organisatorischeeenheid__identificatie

• Fixed missing metadata in CMIS-adapter interface (#925)

• Improved test isolation, reducing Heisenbugs

• Improved display of catalogi without explicit name so that they’re clickable in the admin (#891)

• Fixed broken zaaktype export for published zaaktypen (#964)

Deployment tooling / infrastructure

• Added configuration parameter to opt-in to use X-Forwarded-Host headers to determine the canonical domain of a request to Open Zaak. This is particularly useful when using Istio sidecars for example. (#916)

• Improved dependency management script

• Added CI check to detect improper version bumping

• Bumped version of Django Debug Toolbar to fix an SQL injection. Safe in production, as this dependency is not included in the published Docker images.

• Fixed deleting a Zaak with related documents with CMIS-adapter enabled (#951)

Documentation

• Documented advice to service providers to sign up to the OpenZaak Release Early Notice List and mailing list (#915)

• Updated maturity document (FFPC, #681)

• Improved post-install configuration documentation (#947)

• Documented RabbitMQ’s need for minimum of 256MB RAM

External dependency cleanup

• Dropped nlx-url-rewriter, see manual intervention below

• Dropped drf-flex-fields, it was not used

• Upgraded Django, djangorestframework, djangorestframework-camel-case, drf-yasg & other related packages (#935)

• Replaced django-better-admin-arrayfield fork with upstream again

• Replaced deprecated node-sass (and libsass) with dart-sass (#962)

• Bumped a number of dependencies to their latest release to get security fixes. None of the vulnerabilities appeared to impact Open Zaak, but better safe than sorry.

Warning

Manual intervention required

If you’re upgrading from an older version than 1.2.0 of Open Zaak and using NLX, you need to update to 1.3.5 first, and then update to the 1.4.x series.

In 1.2.0, the configuration of external API’s was reworked, migrating from the nlx-url-rewriter package to zgw-consumers. In 1.4.0, the nlx-url-rewriter package is dropped and no longer present.

1.3.5 (2021-03-25)

1.3.5 is another release focused on bugfixes, performance and quality of life.

Bugfixes

• Bumped cryptography and httplib2 versions, which had some vulnerabilities (#856, #858, #859)

• Fixed an issue where documents were considered external when the CMIS-adapter is enabled (#820)

• Various fixes focused on improving the CMIS-adapter performance (#900, #881, #895)

• Bumped a number of dependencies to stable versions

• Dropped DB constraint preventing versioning of informatieobjecttypen to work as intended (#863)

• Fixed a crash when creating zaaktypen because of too-optimistic input validation (#850)

• Fixed a crash when using invalid query parameters when filtering the list of zaaktypen/ informatieobjecttypen/besluittypen and related objects (#870)

• Mutations in the catalogi admin environment now send notifications similarly to how the same operations in the API would do (#805)

• Fixed filtering ZaakInformatieObjecten with CMIS enabled (#820)

• Fixed a crash when updating Zaaktype.gerelateerdeZaken (#851)

• Fixed incorrect and unexpected Autorisaties API behaviour for applications that are not “ready yet”

  • applications must have either heeftAlleAutorisaties set or have autorisaties related to them (cfr. the standard)

  • applications not satisfying this requirement are not visible in the API (for read, write or delete)

  • applications not satisfying this requirement are flagged in the admin interface and can be filtered

  • when (zaak)typen are deleted, they’re related autorisaties are too. If this leads to an application without autorisaties, the application is also deleted as it is no longer valid

• Fixed serving files for download when using CMIS-adapter and dealing with BytesIO streams in general (#902)

Deployment tooling / infrastructure

• Uses new version of deployment tooling with podman support (alternative to Docker runtime)

• Fixed and improved configuration of the Notifications service in the setup_configuration management command. Generated credentials are now written to stdout and need to be used to configure Open Notificaties (or alternatives).

• Bumped to newer versions of Django and Jinja2, including bug- and security fixes (#906, #907)

Documentation

• Link to the mailing list added to the security documentation

• On the Github issue template you’re now asked to specify which Open Zaak version you’re using

• Updated Standard for Public Code checklist w/r to security procedures (#864)

• Documented the project dependencies with versions < 1.0 (#681)

• Updated the feature request template on Github

• Documented which security-related headers are set by the application and which on webserver level.

• Updated Standard for Public Code checklist w/r to using Open Standards (#679)

New features

• Added support for self-signed certificates, especially where Open Zaak consumes services hosted with self-signed (root) certificates. See the documentation on readthedocs for full details and how to use this. (#809)

Cleanup

• Removed unused and undocumented newrelic application performance monitoring integration

• Updated to pip-tools 6 to pin/freeze dependency trees

1.3.4 (2021-02-04)

A regular bugfix release.

Bugfixes

• Fixed incorrect protocol used in notification payloads (#802)

• Improved test suite determinism (#813, #798)

• Fixed deleting documents when CMIS is enabled (#822)

• Fixed Open Zaak compatibility with an external Documenten API

  • Fixed error logging interpolation (#817)

  • Fixed transaction management (#819)

  • Fixed some django-loose-fk bugs

  • Fixed deleting the remote ObjectInformatieObject on cascading zaak-destroy operations

  • Fixed Besluit.zaak nullable behaviour - now an empty string is returned correctly

• CMIS adapter fixes

  • Implemented Documenten API URL shortening for use with select CMIS DMSs

  • Fixed an oversight where Gebruiksrechten were not updated in the CMIS repository

• Removed notifications for ZIO (partial) update & destroy - the standard only prescribes create notifications.

• Fixed running the test suite with the --keepdb option

• Bumped a number of (frontend) dependencies following Github security notices

• Throw a command error when testing the notifications sending before correctly configuring the Notifications API (#667)

• Fixed Open-Zaak not accepting application/problem+json response media type in content negotation (#577)

• Fixed leaving “producten en diensten” blank in Zaaktype admin (#806)

• Increased the DATA_UPLOAD_MAX_NUMBER_FIELDS Django setting (#807)

• Fixed zaaktype/informatieobjecttype/besluittype publish action API documentation (#578)

• Fixed the handling of the SUBPATH environment variable (#741)

Deployment tooling / infrastructure

• Bumped to version 0.11.1 of the deployment tooling, which added support for:

  • flexibility in certificate configuration

  • enabled http2 in load balancer

  • improved support for additional environment variables

  • Red Hat and CentOS

• Fixed pushing the latest docker image tag to Docker Hub for builds on the master branch

• Open Zaak now provides Helm charts to deploy Open Zaak & Open Notificaties on Haven compliant clusters (thanks to @bartjkdp)

Documentation

• Fixed CI badges in READMEs

• Fixed example recipe for client application developers (#815)

• Documented the security issue process (#831)

• Added Contezza as service provider

• Removed (outdated) documentation duplication in README (#717)

• Removed raven test Sentry test command from documentation - we no longer use Raven but have switched to sentry_sdk instead (#721)

• Documented the need to register notification channels (#666)

• Improved & updated the API schema documentation

• Link to run-time behaviour documentation for each API component (#753)

New features

• Added bulk publishing options to the admin for zaaktype, informatieobjecttype and besluittype (#838)

1.3.3 (2020-12-17)

Security and bugfix release

Warning

this release includes a security fix for CVE-2020-26251, where Open Zaak had a possible vulnerable CORS configuration. It is advised to update as soon as possible. The severity is considered low, since we haven’t been able to actually exploit this due to mitigating additional security configuration in other aspects.

The bugfixes are mostly CMIS-adapter related.

Bugfixes

• The Cross-Origin Resource Sharing configuration is now safe by default - no CORS is allowed. Environment configuration options are made available to make CORS possible to varying degrees, which are all opt-in. This fixes CVE-2020-26251.

• Fixed duplicate ObjectInformatieObject instances being created with CMIS enabled (#778)

• Fixed stale CMIS queryset cache preventing correct chained filtering (#782)

• Fixed some links being opened in new window/tab without norel or noreferrer set in the rel attribute

• Fixed multiple EnkelvoudigInformatieobject instances having the same bronorganisatie and identificatie (#768). If you’re not using the CMIS-adapter, see the manual intervention required below.

• Fixed a bug retrieving ObjectInformatieObject collection in the Documenten API when CMIS is enabled. This may also have affected the Gebruiksrechten resource. (#791)

Documentation

• Improved documentation for CMIS services configuration

• Fixed a typo in the Governance document

• Documented environment variable to disable TLS certificate validation. This should never be used in production, instead the certificate setup should be fixed.

Other changes

• Enabled CMIS-adapter logging in DEBUG mode

• Migrated CI from Travis CI to Github Actions

• Explicitly test PostgreSQL versions 10, 11 and 12 (#716)

• Optimized CI build to re-use Docker image artifacts from previous jobs

• Replaced postman.io mocks subscription with nginx container (#790)

• Avoid some unnecessary queries when CMIS is enabled

• Implemented a (likely) fix to non-deterministic behaviour in the test suite (#798)

Warning

Manual intervention required.

There is a chance that documents have been created in the Documents API with duplicate (bronorganisatie, identificatie) combinations.

We’ve provided a management command to check and fix these occurrences.

Run python src/manage.py detect_duplicate_eio --help in an Open Zaak container to get the command line options. By default, the command is interactive:

single-server

Kubernetes

$ docker exec openzaak-0 src/manage.py detect_duplicate_eio
Checking 30 records ...
Found no duplicate records.

$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
cache-79455b996-jxk9r       1/1     Running   0          2d9h
nginx-8579d9dfbd-gdtbf      1/1     Running   0          2d9h
nginx-8579d9dfbd-wz6wn      1/1     Running   0          2d9h
openzaak-7b696c8fd5-hchbq   1/1     Running   0          2d9h
openzaak-7b696c8fd5-kz2pb   1/1     Running   0          2d9h

$ kubectl exec openzaak-7b696c8fd5-hchbq -- src/manage.py detect_duplicate_eio
Checking 30 records ...
Found no duplicate records.

1.3.2 (2020-11-09)

Open Zaak 1.3.2 fixes a number of issues discovered in 1.3.1. Note that there are two manual interventions listed below these patch notes. Please read them before updating.

Changes

• Added messages in the admin if the selectielijst configuration is invalid (#698)

• Applied a unique constraint on user e-mail address (if provided) (#589) - see manual intervention warning below.

• Upgraded to a newer version of zgw-consumers, dropping the extra configuration field for services (#710)

• Implemented the upstream API bugfix, adding some zaken list query filters (https://github.com/VNG-Realisatie/gemma-zaken/issues/1686, #732)

• Added Github’s code-scanning to detect vulnerable code patterns

• Updated frontend dependencies to secure versions

• Updated backend and deployment dependencies to secure versions (notably cryptography) (#755, #756)

• [CMIS-adapter] Changed EnkelvoudigInformatieobject.identificatie generation. CMIS query does not (always) support LIKE queries, nor does it support aggregation queries (#762)

Bugfixes

• Fixed #711 – changed Rol.omschrijving max_length from 20 -> 100

• Fixed input validation of binary document content (when the client forgets to base64 encode it) (#608)

• Fixed primary keys being localized in admin URLs (#587)

• Fixed a crash when trying to download non-existant informatieobjecten (#584)

• Corrected validation of Eigenschap.lengte. API and admin are now consistent, and decimals are now correctly interpreted (comma instead of dot) (#685)

• Fixed the register_kanaal management command auth-issue (#738)

• Fixed a bug where deleted zaaktypen had dangling Autorisatie records (#713) - see manual intervention warning below.

• Updated to CMIS adapter 1.1.1 to fix some bugs (#760)

Documentation

• Update Governance.md after a number of steering group meetings

• Clarified that Ansible Galaxy roles and collections need to be installed separately

• Added a (technical) roadmap draft

• Drafted code style/code architecture principles

• Fixed a mix-up between authorizations/authentications API (#722)

• Docker image badge now points to Docker Hub

• Removed mention of Klantinteractie-API’s - it’s unclear what’s being done with these API’s

• Started documentation entries for developers of client/consumer applications

Warning

Manual intervention required.

E-mail addresses are used for logging in to the admin environment, which had no unique constraint. This is corrected in a database migration, which will crash if there are users with duplicate e-mail addresses. You should fix the duplicate addresses BEFORE updating.

Warning

Manual intervention required.

Some cleanup is required because of a synchronization bug. You need to run the following sync_autorisaties management command.

single-server

Kubernetes

docker exec openzaak-0 src/manage.py sync_autorisaties

$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
cache-79455b996-jxk9r       1/1     Running   0          2d9h
nginx-8579d9dfbd-gdtbf      1/1     Running   0          2d9h
nginx-8579d9dfbd-wz6wn      1/1     Running   0          2d9h
openzaak-7b696c8fd5-hchbq   1/1     Running   0          2d9h
openzaak-7b696c8fd5-kz2pb   1/1     Running   0          2d9h

$ kubectl exec openzaak-7b696c8fd5-hchbq -- src/manage.py sync_autorisaties

1.3.1 (2020-08-31)

Changes

• Updated CMIS-adapter to 1.1 featuring support CMIS 1.0 Webservice binding and various new configuration options.

• Added support for configurable Selectielijst years to retrieve specific years from the Selectielijst API (#689)

• Prevent error monitoring from logging special personal data (#696)

Bugfixes

• Accept comma separated in EigenschapSpecificatie.waardenverzameling (#686)

Documentation

• Added SPDX license headers and check.

• Added Docker storage hint to make sure users run the Docker containers on volumes with enough disk space.

1.3.0 (2020-07-29)

Version 1.3.0 of Open Zaak introduces some new features, quality of life changes and fixes bugs discovered in 1.2.0.

There is no 1.2.1 bugfix release. Upgrading from 1.2.0 to 1.3.0 requires no manual intervention.

What’s new?

• Added experimental support for CMIS backends for the Documenten API, as an alternative to Open Zaak database + filesystem. See the documentation for more details.

• Added a feature flag to allow unpublished *Typen to be used. This should only be used in Proof-of-concept environments, as it violates the VNG standard.

• Added a number of CLI commands for initial Open Zaak setup following installation. See the documentation for more details.

• Implemented extra zaak_list filters, added in 1.0.2 of the Zaken API standard

  • maxVertrouwelijkheidaanduiding

  • betrokkene

  • betrokkeneType

  • omschrijvingGeneriek

  • natuurlijk persoon BSN

  • medewerker identificatie

Bugfixes and general QOL changes

• Positioned the Foundation for Public Code and checked Open Zaak against their standard/guidelines

• The documentation now includes a Public Code checklist

• Added Code of Conduct

• Added Governance documentation

• Fixed running tests with --keepdb option

• Fixed the admin form for Zaaktype-Informatieobjecttype relation

• Fixed importing a Zaaktype-Informatieobjecttype with a Statustype relation

• Improved documentation for deploying on Kubernetes

• Added English version of README

• Fixed configuration form for external services when the NLX directory has not been configured (yet)

• Fixed BesluitType create in the admin (#594)

• Added and documented performance-profiling tooling for Open Zaak developers

• Fixed performance regression in zaak_list endpoint operation :zap:

• Fixed a crash on malformed UUIDs in endpoint URLs that expect a valid UUID 4 pattern

• Added the environment configuration reference to the published documentation

• Refactored notifications/selectielijst configuration to use the external services configuration

• Fixed EigenschapSpecificatie.waardenverzameling default value (empty list) (#611)

• Fixed missing validation on (zaaktype, eigenschapnaam) uniqueness

• Added Slack invite link

• Relaxed Resultaat.afleidingswijze validation in the admin too (see also 6e38b865c)

• Improved “Contributing” section

1.2.0 (2020-04-20)

New feature release and a set of bugfixes included.

Features

• Update admin layout version

• #507 – use the original filename when downloading a document from the admin

• Reworked configuration of external APIs

• Added option to specify your NLX outway location and network

• Added the ability to enable/disable APIs offered by Open Zaak

• Added the option to configure external APIs, optionally selecting services from the NLX network.

• Added support for custom OAS urls. Note that you need to add them manually in zgw_consumers.Service for existing APIs (you can do it in the admin).

Bugfixes

• Bumped a number of libraries to their latest security releases

• #511 – fix saving of resultaattype if bewaartermijn is null

• #495 – use correct page titles for api schemas per component

• #318 – Fixed (BesluitType)Admin M2M relations so that they show content from the same catalogus only

• Fixed Document inhoud base64 validation

• Enabled pre-filling the informatieobjecttype in zaaktype-informatieobjecttype admin

• #532 – fixed issue with Resultaattype.omschrijving_generiek not updating

• #551 – ensure client credentials are deleted when an Applicatie is deleted in in the admin

• #543 – fix error when trying to create a document in the admin

• Fixed creating a Zaaktype with partial referentieProces gegevensgroep

• #553 – made Eigenschap.specificatie required in admin

• #557 – fix handling of brondatumArchiefProcedure: null

• #558 – fixed ZaakBesluit DELETE calls

• #556 – fixed admin crash for resultaattype when the related zaaktype does not have a selectielijst procestype set

• #559 – fixed deploying Open Zaak on a subpath (as opposed to on its own (sub)domain)

• #554 – fixed admin crash when related informatieobjecttypen/besluiten are not available for a given zaak.

• #562 – fixed nested Eigenschap.specificatie being ignored

Documentation

• Documentation minimal version of required development tooling

• #299 – Fixed notification documentation generation

• Updated PR template

• #534 – updated documentation links in the API specs

1.1.1 (2020-03-13)

Bugfix release w/r to deployment and ADFS

• Added option to disable group sync in ADFS login. If the ADFS provider does not provide the group claim, this would otherwise reset the user groups you carefully configured.

• Updated single-server deployment to make sure the web-server can read and serve uploaded files through the Documenten API.

1.1.0 (2020-03-11)

New feature release. Note that this is not yet an implementation of the 1.1.x API specs!

• Included playbooks for NLX deployment

• Added communication channels to the docs (i.e. - how to find/contact us!)

• Added ADFS support (i.e. you can now log in to the admin with ADFS)

• Fixed some deployment tooling

1.0.4 (2020-03-05)

Improved support for integration with other APIs, most notably BAG/BRT APIs from the kadaster (see https://pdok.nl). This increases the usability of ZaakObject relations.

• Added api-test.nl badge - proves that Open Zaak is compliant with the API’s voor zaakgericht werken standard

• Added small documentation improvements

• Updated notification setup instructions

• Added support for API authentication with a simple API key (such as BAG or BRT)

• Added support for URL transformation so that data-fetching is forced over NLX

1.0.3 (2020-02-25)

Fixed infrastructure on single-server where Open Zaak and Open Notificaties run on the same machine.

1.0.2 (2020-02-19)

Bugfixes and usability improvements

• Improve selectielijst-resultaten display in ResultaatType admin (#480)

• Improved production description

• Fixed file permissions for installation on single-server (#481)

1.0.1 (2020-02-17)

Bugfixes from initial release

• Added version information to Docker image

• Added better admin validation in various places [prevent crashes]

• Updated some documentation

• Fixed Besluiten API spec defects

• Fixed rendering the admin detail pages for read-only resources

• Fixed the cache for resultaattypeomschrijvinggeneriek

• Updated to latest Django security release

• Improved help-text for read-only fields

• Fixed CI

1.0.0 (2020-02-06)

🎉 First release of Open Zaak.

Features:

• Zaken API implementation

• Documenten API implementation

• Catalogi API implementation

• Besluiten API implementation

• Autorisaties API implementation

• Support for external APIs

• Admin interface to manage Catalogi

• Admin interface to manage Applicaties and Autorisaties

• Admin interface to view data created via the APIs

• NLX ready (can be used with NLX)

• Documentation on https://open-zaak.readthedocs.io/

• Deployable on Kubernetes, single server and as VMware appliance

• Automated test suite

• Automated deployment

Getting started

The project is developed in Python using the Django framework. There are 3 sections below, focussing on developers, running the project using Docker and hints for running the project in production.

Installation

Prerequisites

You need the following libraries and/or programs:

• Python 3.7 or above

• Python Virtualenv and Pip 20.0 or above

• PostgreSQL 10.0 or above, with the PostGIS-extension

• Node.js 10.0 or above

• npm 6.0 or above

• Docker 19.03 or above (and docker-compose)

Step by step

Developers can follow the following steps to set up the project on their local development machine.

1. Navigate to the location where you want to place your project.

2. Get the code:

   $ git clone git@github.com:open-zaak/open-zaak.git
   $ cd open-zaak
   

3. At this point you can already built the Docker image and run Open Zaak. You can skip this if you don’t want that.

   $ docker-compose up
   

   Note: If you are using Git on Windows, line-endings might change in checked out files depending on your core.autocrlf setting in .gitconfig. This is problematic because files are copied into a Docker image, which runs on Linux. Specifically, the bin/docker_start.sh file is affected by this which causes the Docker container fail to start up.

4. Install all required libraries:

   $ virtualenv env  # or, python -m venv env
   $ source env/bin/activate
   $ pip install -r requirements/dev.txt
   

5. Install the front-end CLI tool gulp if you’ve never installed them before and install the frontend libraries:

   $ npm install
   $ npm run build
   

6. Activate your virtual environment and create the statics and database:

   $ source env/bin/activate
   $ python src/manage.py migrate
   

7. Create a superuser to access the management interface:

   $ python src/manage.py createsuperuser
   

8. You can now run your installation and point your browser to the address given by this command:

   $ python src/manage.py runserver
   

Note: If you are making local, machine specific, changes, add them to src/openzaak/conf/includes/local.py. You can also set certain common variables in a local .env file. You can base these files on the example files included in the same directory.

Note: You can run watch-tasks to compile Sass to CSS and ECMA to JS using npm run watch. By default this will compile the files if they change.

Update installation

When updating an existing installation:

1. Activate the virtual environment:

   $ cd open-zaak
   $ source env/bin/activate
   

2. Update the code and libraries:

   $ git pull
   $ pip install -r requirements/dev.txt
   $ npm install
   $ npm run build
   

3. Update the statics and database:

   $ python src/manage.py migrate
   

Testsuite

To run the test suite:

$ python src/manage.py test openzaak

Configuration via environment variables

A number of common settings/configurations can be modified by setting environment variables, add them to your .env file or persist them in src/openzaak/conf/includes/local.py.

• SECRET_KEY: the secret key to use. A default is set in dev.py

• DB_NAME: name of the database for the project. Defaults to open-zaak.

• DB_USER: username to connect to the database with. Defaults to open-zaak.

• DB_PASSWORD: password to use to connect to the database. Defaults to open-zaak.

• DB_HOST: database host. Defaults to localhost

• DB_PORT: database port. Defaults to 5432.

• SENTRY_DSN: the DSN of the project in Sentry. If set, enabled Sentry SDK as logger and will send errors/logging to Sentry. If unset, Sentry SDK will be disabled.

Testing with CMIS adapter enabled

The tests for Open Zaak with the CMIS adapter enabled use Alfresco as the Document Management System. This is run with docker compose, using the compose file docker-compose.travis-cmis.yml. In the folder extension/, there are xml files defining the custom models used. The file config/alfresco-global.properties contains the global properties that are required to run Alfresco.

The containers are launched with:

$ docker-compose -f docker-compose.travis-cmis.yml up -d

This creates a volumes/ directory where all the logs and the data are stored.

You can run only the CMIS tests with:

$ python src/manage.py test src --tag cmis

In order to choose whether to test with the browser binding or the web service binding, the environment variable CMIS_BINDING can be set either to BROWSER or WEBSERVICE.

Or run the test-suite without CMIS tests:

$ python src/manage.py test src --exclude-tag cmis

Settings

All settings for the project can be found in src/openzaak/conf. The file includes/local.py overwrites settings from the base configuration, and is only loaded for the dev settings.

Principles and code style (draft)

Definining (architectural) principles and code style keeps the code base consistent and manages expectations for contributions.

Backend

On the backend, we use the Django framework and follow the project structure of having apps within the project.

• Django apps contains models, views and API definitions. They group a logical part of the greater project which is loosely coupled to other apps.

  Tests are in the django app package. Split tests in logical modules, and try to avoid complex nesting structures.

• All apps must go in the src/openzaak directory, which namespaces all the Open Zaak code in the openzaak package. This prevents name conflicts with third party applications.

• Application names should always be in plural form.

• Components from the API standard go in openzaak.components.<component>.

• Settings go in openzaak.conf, which is split according to deploy environment:

  • dev

  • ci (Travis)

  • staging

  • production

  • docker

  Settings must always be defined in the openzaak.conf.includes.base or openzaak.conf.includes.api with sane defaults.

• Global runtime Open Zaak configuration (database backed) should go in the openzaak.config app.

• Generic tools that are used by specific apps should be a openzaak sub-package, or possibly go in openzaak.utils.

• Integration with other, third-party services/interfaces should go in a openzaak.contrib package. This is currently (!) not the case yet.

• Code style is enforced in CI with black. When Black is not conclusive, stick to PEP 8.

• Imports are sorted using isort.

Frontend

• Javascript goes in the src/openzaak/js directory

• If possible, stick to Vanilla JS

• (Highly) dynamic interfaces are built using React. Components should:

  • go in src/openzaak/js/components

  • preferably use functional components with react hooks

  • aim to be prop-driven and keep as little local state as possible

  • be properly prop-type annotated

  • have default values for optional props

• Code should be linted and quality checked using ESLint, with the AirBnB preset

• Browser support: latest and latest -1 version of the major browsers

Release process

Open Zaak makes use of quite a bit of Continuous Integration tooling to set up a full release process, all driven by the Git repository.

Travis CI

Our pipeline is mostly implemented on Travis:

Merges to the master branch are built on Travis, where:

1. Tests are run

2. Code quality checks are run

3. Compatibility with the API’s voor zaakgericht werken standard spec is tested

4. The Docker image is built and published

If the build is for a Git tag on the master branch, then the image is built and publish with that version tag to Docker Hub.

Releasing a new version

Releasing a new version can only be done by people with merge permissions to the master branch belonging to the Open Zaak organisation on Github.

Assuming a current version of 0.9.0:

Create a release branch

git checkout -b release/1.0.0

Update the changelog

Update CHANGELOG.rst in the root of the project, and make sure to commit the changes.

Bump the version

bumpversion minor

and commit:

git commit -am ":bookmark: Bump version to 1.0.0"

Push the changes and make a pull request.

Tag the release

Once the PR is merged to master, check out the master branch and tag it:

git checkout master
git pull
git tag 1.0.0

Tagging will ensure that a Docker image openzaak/open-zaak:1.0.0 is published.

Releasing a dev-version

You can also make use of bumpversion to mark a release as a dev release:

bumpversion dev

Which will take care of the major.minor.patch.devX suffix of devX.

Performance

Performance is measured using different types of scenarios.

Performance profiling

Since Open Zaak emphasizes good performance, we need tooling to measure performance. There is no point in optimizing without knowing what you’re optimizing for. Every optimization should be preceded by a base-line profiling run, which provides the information on where to optimize.

Note

The tooling discussed here is only enabled with the development settings. You should verify the results on the same dev-environment, as this keeps the hardware and configuration consistent.

Profiling plain HTML pages

Plain HTML pages follow the classic pattern of a request that gets rendered into a template response, which is eventually displayed in the browser.

In development mode, Django Debug Toolbar (DjDT) is enabled by default. It provides a side-panel with information on timings, SQL queries and where they originate.

Profiling API endpoints

DjDT is not suitable for API endpoints, because there is no HTML body being rendered, thus the panel cannot be displayed. We use Django Silk instead for endpoint profiling.

Django silk is not enabled by default, and you can only use it with the development settings. To enable profiling, set the environment variable PROFILE to a truthy value, e.g.:

PROFILE=yes DEBUG=no python src/manage.py runserver

The profile data is available on http://localhost:8000/silk/. You can make the API calls using Postman, and they’ll show up in the Silk dashboard.

Silk provides information on total request time, how many and which SQL queries ran, timings of the queries and what caused the queries to run.

General recommendations

Disable DEBUG mode

You should use DEBUG=no when profiling, which disables some Django internals that can degrade performance and thus give skewed results. Most notably, in debug mode, Django keeps track of the database queries in an array.

Queries are usually the bottleneck

In most web-apps, the database is the bottleneck for performance. Large amounts of queries, or big/complex queries should catch your attention. Especially repeating queries are suspicious - often they can be mitigated using select_related or prefetch_related.

Measure relative performance

Expressing performance gains in percentage is usually the most useful. Of course, endpoints taking 5 or 10 seconds in total are both bad, so use common sense when interpreting the results. However, it’s interesting if you can attribute a 20% total request time reduction and 90% less queries to a single change.

If you have profiling numbers in production, you can apply the relative gains in performance to the production numbers to get a ballkpark figure of the production performance.

Measuring performance

Goal

The purpose of performance measurements is to gain insight into the relationships between system requirements, number of users and response times of the API’s. From these relationships we can draw up the minimum system requirements, depending on the number of users, on which the API’s still perform acceptable.

A standardized performance measurement also provides insight into which effect various optimizations have.

Technical test scenarios

The performance of some often used API-calls (requests) is measured to gain insights in the technical performance.

Zaken API

1. Retrieve ZAAKen (GET /api/v1/zaken)

2. Retrieve ZAAK (GET /api/v1/zaken/d4d..2e8)

3. Create ZAAK (POST /api/v1/zaken)

Catalogi API

1. Retrieve ZAAKTYPEn (GET /api/v1/zaaktypen)

2. Retrieve ZAAKTYPE (GET /api/v1/zaaktypen/d4d..2e8)

3. Create ZAAKTYPE (POST /api/v1/zaaktypen)

Besluiten API

1. Retrieve BESLUITen (GET /api/v1/besluit)

2. Retrieve BESLUIT (GET /api/v1/besluit/d4d..2e8)

3. Create BESLUIT (POST /api/v1/besluit)

Documenten API

1. Retrieve ENKELVOUDIGINFORMATIEOBJECTen (GET /api/v1/enkelvoudiginformatieobjecten)

2. Retrieve ENKELVOUDIGINFORMATIEOBJECT (GET /api/v1/enkelvoudiginformatieobjecten/d4d..2e8)

3. Create ENKELVOUDIGINFORMATIEOBJECT (POST /api/v1/enkelvoudiginformatieobjecten)

Test specification

Using scenarios

A scenario in this test specification is equal to an API-call (request). Each API resource is continuously without any delay, or waiting time, between each request. This way, we can determine the maximum number of requests per second and average response times.

Virtual users

We test with an increasing number of virtual users, from 1 to 100, that concurrently execute the test scenarios. A virtual user is technically a script that executes the different scenarios one after another. This way, we see the number of virtual users that concurrently access the API’s and its impact on performance.

Testdata

The testset that is present in the database:

• 1.000.000 Zaken in the Zaken API

• 1.000.000 Documenten in the Documenten API

• 1.000.000 Besluiten in the Besluiten API

• 1 Catalogus with 100 Zaaktypen in the Catalogi API

Functional test scenarios

Performance testing is done by accessing the APIs as if they were being used by an application, or a virtual system. Some typical functional scenarios have been observed in similar systems:

1. Retrieve Zaken overview

2. Retrieve Zaken overview for a specific Zaaktype

3. Search Zaken by location

4. Search Zaken by person

5. Retrieve Zaak details

6. Retrieve history

7. Create Zaak

8. Add Status

9. Add Betrokkene

10. Add Document

11. Add Besluit

12. Add Resultaat

Scenario’s in API-verzoeken

All functional scenarios have been translated into API requests. The number of API requests, the exact query parameters for filtering and/or sorting lists, and the data sent for creating objects are all very dynamic in practice. For each functional scenario, one or more specific API requests are prepared that fill in the scenario as well as possible.

A number of API requests have been placed out of scope because they are not part of the APIs voor Zaakgericht werken but are most likely required to build a functional user interface.

Retrieve Zaken overview (1)

Retrieve an unfiltered list of Zaken, along with their Zaaktype and Statustype.

Zaken API

• 1x ZAAKen retrieve (GET /api/v1/zaken)

• 1x STATUSsen retrieve (GET /api/v1/statussen)

Catalogi API

• 1x ZAAKTYPEn retrieve (GET /api/v1/zaaktypen)

• 1x STATUSTYPEn retrieve (GET /api/v1/statustypen)

Retrieve Zaken overview for a specific Zaaktype (2)

Retrieve an unfiltered list of Zaken, along with their Zaaktype and Statustype. Each Status is requested, filtered on their matching Statustype for the appropriate Zaaktype.

Zaken API

• 1x ZAAKen retrieve (GET /api/v1/zaken?zaaktype=/api/v1/zaaktypen/011..3c1)

• 3x STATUSsen retrieve (GET /api/v1/statussen?statustype=/api/v1/statustypen/f82..396)

Catalogi API

• 1x ZAAKTYPEn retrieve (GET /api/v1/zaaktypen/011..3c1)

• 1x STATUSTYPEn retrieve (GET /api/v1/statustypen?zaaktype=/api/v1/zaaktypen/011..3c1)

Search Zaken by location (3)

Retrieve a list of Zaken that match a specific geographical area (polygon).

Zaken API

• 1x ZAAKen zoeken (POST /api/v1/zaken/_zoek)

Search Zaken by person (4)

Retrieve a list of Zaken that match a specific person (Betrokkene).

• 1x Betrokkene zoeken (buiten scope)

Zaken API

• 1x ZAAKen filteren GET /api/v1/rollen?betrokkene=https://personen/api/v1/a66c38

Retrieve Zaak details (5)

Retrieve a complete Zaak, with its Resultaat, Besluit, 3 Documenten, 2 Zaakobjecten and 3 Betrokkenen.

• 3x Betrokkenen retrieve via ROLlen-resultaat (buiten scope)

• 2x Objecten retrieve via ZAAKOBJECTen-resultaat (buiten scope)

Zaken API

• 1x ZAAK retrieve (GET /api/v1/zaken/d4d..2e8)

• 1x STATUSsen retrieve (GET /api/v1/statussen?zaak=/api/v1/zaken/d4d..2e8)

• 1x RESULTAAT retrieve (GET /api/v1/resultaten/f84..e9e)

• 1x ROLlen retrieve (GET /api/v1/rollen?zaak=/api/v1/zaken/d4d..2e8)

• 1x ZAAKOBJECTen retrieve (GET /api/v1/zaakobjecten?zaak=/api/v1/zaken/d4d..2e8)

Catalogi API

• 1x ZAAKTYPE retrieve (GET /api/v1/zaaktypen/011..3c1)

• 1x STATUSTYPEn retrieve (GET /api/v1/statustypen?zaaktype=/api/v1/zaaktypen/011..3c1)

• 1x BESLUITTYPE retrieve (GET /api/v1/besluittypen?zaaktype=/api/v1/zaaktypen/011..3c1)

• 1x RESULTAATTYPE retrieve (GET /api/v1/resultaattypen/712..a7c?zaaktype=/api/v1/zaaktypen/011..3c1)

Documenten API

• 1x OBJECTINFORMATIEOBJECTen retrieve (GET /api/v1/objectinformatieobjecten?object=/api/v1/zaken/d4d..2e8)

• 3x ENKELVOUDIGINFORMATIEOBJECT retrieve (GET /api/v1/enkelvoudiginformatieobjecten/cd6..d90)

Besluiten API

• 1x BESLUITen retrieve (GET /api/v1/besluiten?zaak=/api/v1/zaken/d4d..2e8)

Retrieve history (6)

Retrieve the combined audit trails of a Zaak, Besluit and 3 Documenten from their respective API’s.

Zaken API

• 1x AUDITTRAIL retrieve (GET /api/v1/zaken/d4d..2e8/audittrail)

Documenten API

• 3x AUDITTRAIL retrieve (GET /api/v1/enkelvoudiginformatieobjecten/cd6..d90/audittrail)

Besluiten API

• 1x AUDITTRAIL retrieve (GET /api/v1/besluiten/a28..6d3/audittrail)

Create Zaak (7)

Create a Zaak with its initial Status and a Rol with the person that initiated the Zaak.

Zaken API

• 1x ZAAK create (POST /api/v1/zaken)

• 1x STATUS create (POST /api/v1/status)

• 1x ROL create (POST /api/v1/rollen)

Add Status (8)

Zaken API

• 1x STATUS create (POST /api/v1/status)

Add Betrokkene (9)

• 1x Persoon zoeken (buiten scope)

Zaken API

• 1x ROL create (POST /api/v1/rollen)

Add Document (10)

Create a Document and make a relation to a Zaak.

Zaken API

• 1x ZAAK-INFORMATIEOBJECT create (POST /api/v1/zaakinformatieobjecten)

Documenten API

• 1x ENKELVOUDIGINFORMATIEOBJECT create (POST /api/v1/enkelvoudiginformatieobjecten)

Add Besluit (11)

Besluiten API

• 1x BESLUIT create (POST /api/v1/besluiten)

Add Resultaat (12)

Zaken API

• 1x RESULTAAT create (POST /api/v1/resultaten)

Test specificatie

Gebruik van scenario’s

Not every scenario is executed as often. For example, a Zaak is requested more often than it is created. In the table below, for every 20 x “Retrieve Zaak details” scenario, 10x “Create Zaak” is executed. This was then converted to a percentage, assuming that all scenarios represent 100%. We call this the weight.

To simulate a user perspective, a specific waiting time is introduced after each scenario. The waiting time represents, for example, the time a user needs to enter data into the user interface or the time it takes to interpret the data. In the table below, after executing “Create Zaak”, the script waits between 0 and 10 minutes (5 minutes on average).

Finally, the number of API-calls is shown for each scenario.

#	Scenario	Weight		Wait time (m)		API-calls
.	.	Abs.	%	Avg.	Range	.
1	Retrieve Zaken overview	20	20%	2	0 - 4	4
2	… for a specific Zaaktype	10	10%	2	0 - 4	6
3	Search Zaken by location	1	1%	1	0 - 2	1
4	Search Zaken by person	10	10%	1	0 - 2	1
5	Retrieve Zaak details	8	8%	2	0 - 4	14
6	Retrieve history	2	2%	3	0 - 6	5
7	Create Zaak	10	10%	5	0 - 10	3
8	Add Status	20	20%	2	0 - 4	1
9	Add Betrokkene	3	3%	3	0 - 6	1
10	Add Document	12	12%	4	0 - 8	2
11	Add Besluit	2	2%	3	0 - 6	1
12	Add Resultaat	2	2%	3	0 - 6	1
.	Total	100	100%	31		40

If taken the weight into account, the overall average waiting time is 5 minutes and the average number of requests per scenario is 7.

Virtual users

We test with an increasing number of virtual users, from 1 to 100, that concurrently execute the test scenarios. A virtual user is technically a script that executes the different scenarios one after another, in the distribution and with the associated waiting time as shown in the table above.

Testdata

The testset that is present in the database:

• 1.000.000 Zaken in the Zaken API

• 1.000.000 Documenten in the Documenten API

• 1.000.000 Besluiten in the Besluiten API

• 1 Catalogus with 100 Zaaktypen in the Catalogi API

Apache Benchmark stress test

Apache Bench is a good tool to measure raw HTTP performance. It is not indicative for application performance, but rather aimed at testing your infrastructure.

This document describes the configuration to run ab to make meaningful comparisons and see which changes have which effect.

Running ab

ab -n 1000 -c 100 https://test.openzaak.nl/zaken/api/v1/

The most interesting statistic is the number of requests per second, which you want to be as high as possible.

Philosophy

We run a total of 1000 requests to measure performance, using up to 100 concurrent users. Playing around with the concurrency level can reveal hot spots or ideal numbers.

We test and API endpoint that has no IO and is inexpensive to generate. We don’t want to stress the application with queries or complicated template rendering, which is why we pick an API root that generates very little JSON.

Note that if you are running the tests directly against other services, that you may have to specify the Host header: -H 'Host: localhost', otherwise Django will generate an HTTP 400 because of the InvalidHostHeader, which is very fast but not representative.

Also note that you want to run ab from a fast client machine. Aim for a server in a different data center. We’ve seen cases where our client is limited in requests per second because of a slow home network.

Parameters to vary

• Traefik/Ingress: SSL termination happens here. We’ve seen Traefik CPU spikes with large private keys (4096 bits), and performance did improve a bit by changing to EC 256 bits at the cost of a worse SSL rating (A+ to A).

• Number of nginx replicas - more replicas can handle more traffic from Traefik

• Number of Open Zaak replicas - by default each container can handle 4 concurrent requests (2 process * 2 threads per process)

• Number of Open Zaak processes/threads in a container

• CPU/memory resource requests/limits in Kubernetes. Assigning more CPU/memory power will probably give better results. This applies for the Traefik, nginx and Open Zaak containers.

Security measures in Open Zaak

The following is a non-exhaustive list of configurations in Open Zaak to enhance security.

nginx template considerations

When deploying Open Zaak on a VM, nginx is used as a reverse proxy. A number of headers are set in the virtual host:

Referrer-Policy: "same-origin";

the HTTP_REFERER header is sent only to Open Zaak pages

X-Content-Type-Options: "nosniff";

protects against user-uploaded content, which is relevant because of the Documents API and serving of that user content

X-XSS-Protection: "1; mode=block";

note that this is not honored by most browsers anymore, but it doesn’t hurt to include it

Content-Security-Policy

opt-in, configure the deployment playbook accordingly

Feature-Policy: "autoplay 'none'; camera 'none'" always;

there’s no need for these :-)

Open Zaak settings

X-Frame-Options is set to DENY

no (i)frames are allowed at all

How to’s

When you’re developing on Open Zaak, some things aren’t as obvious as they could be. The HOW TO section of the documentation should give you some quick insight on a number of topics.

Dependencies

Backend dependencies are managed with pip-tools. pip-compile (part of pip-tools) takes *.in files with version constraints, and outputs the entire transitive dependency tree to *.txt files.

The backend dependencies are layered:

• requirements/base.txt: the minimal set of packages that is needed to run Open Zaak in a production-like environment

• requirements/ci.txt: base.txt + any testing/CI tools to guard the quality of Open Zaak

• requirements/dev.txt: ci.txt + developer tools only installed in a local environment to develop Open Zaak itself

Adding a backend dependency

Sometimes new features require new dependencies that aren’t used yet in this project.

1. Open requirements/base.in

2. Add the dependency in the right logical group

3. Run ./bin/compile_dependencies.sh

4. Commit the three changed requirements/*.txt files

For CI or development dependencies you add them to requirements/test-tools.in or requirements/dev.in respectively.

Upgrading a backend dependency

It happens that existing backend dependencies need to be upgraded (bugfixes, security releases…). This is also done through ./bin/compile_dependencies.sh. Any extra arguments supplied are forwarded to the underlying pip-compile calls.

1. Determine which package needs to be upgraded, for example Django

2. run ./bin/compile_dependencies.sh -P django (substitute with appropriate package name)

3. Commit the changed requirements/*.txt files

This works for base, ci and dev dependencies.

Note

You can constraint versions, such as -P django~=2.2.0 to get the latest patch version, or -P djangorestframework<3.13 for example.

If you’re looking for developer documentation as an Open Zaak client, head over to Open Zaak client documentation!