In this paper we address an exact algorithm for the block relocation problem with a stowage plan. This problem is an extension of the container (block) relocation problem that aims at minimizing the total number of relocations required for retrieving containers piled up in a container yard. The difference is that a stowage plan in a vessel is given in advance and it is taken into consideration when retrieving containers. More specifically, the retrieval order of containers should be feasible with respect to a given stowage plan. We construct a branch-and-bound algorithm with iterative deepening for unrestricted and restricted variants of this problem. To this end, we propose three lower bounds on the total number of relocations. We also extend dominance rules for the container relocation problem, in order to restrict the search space and improve the search efficiency. The effectiveness of the proposed algorithm is examined by numerical experiments for benchmark instances from the literature.