The algtype-based test broke when algtype got a bit more fine-grained, so replace with an explicit check for the invalid key types. R=ken2 CC=golang-dev https://golang.org/cl/5071041
