I went to Teledahn first and finished it and took the stone to Gahreesen. I hadn't set foot in Gahreesen prior to that.
Now I cannot go anywhere other than the cells. The book in the Relto shelf...