One thing I've really disliked is the fact that contrib modules had installation scripts that enforced the schema to public for the installation. In my opinion, for no useful reason at all.
For example, I often install the pgcrypto contrib module. And I install this in the pgcrypto schema, that I then either add to the search_path variable or just explicitly use in my queries, with things like pcrypto.crypt('foobar','barfoo'). For versions prior to 9.1, being able to do this required me to manually edit the installed pgcrypto.sql file, to remove the SET search_path = public; command.
Extensions in 9.1 makes this so much nicer. To get pgcrypto into it's own schema, I now just need to do:
postgres=# CREATE SCHEMA pgcrypto; CREATE SCHEMA postgres=# CREATE EXTENSION pgcrypto SCHEMA pgcrypto; CREATE EXTENSION
If I happen to create it in public by mistake, I can even move it after the fact!
postgres=# ALTER EXTENSION pgcrypto SET SCHEMA pgcrypto; ALTER EXTENSION
You still need to create the schema manually - in theory we could auto-create that, but the work is still a lot easier than before. And fully supported!