Minecraft was made so you were able to break blocks. If you break a block on a map, it disappears in your client but the server notices that that certain block is missing, and therefore puts it back. They can't change it to adventure mode, because the players still need to be able to break leaves, mushrooms, grass, flowers and such and that wouldn't be possible anymore in game mode 3. Also, they can't modify adventure mode itself because it is in your minecraft client, which is something else than the server.