I've eventually found a solution. The following script has to be included:
<script type='text/javascript' src='/canvas/sdk/js/publisher.js'></script>
Then use this line to close the Quick Action window:
Sfdc.canvas.publisher.publish({ name: "publisher.close", payload:{ refresh: "true" }});