This issue is solved now.

The cause of the error was that I used the DotNetNuke 5 gzip compression setting.

This setting has caused AJAX issues before, and is removed in DNN7.

IIS compression does not cause the problem.