Urban flow fields computed by two steady Computational Fluid Dynamics models based on the Reynolds-averaged Navier Stokes equations (CFD-RANS) are compared to validation data measured in a boundary-layer wind-tunnel experiment. The numerical simulations were performed with the research code ADREA and the commercial code STAR-CD. Turbulent flow within and above a 1:225-scale wind-tunnel model representing a novel semi-idealized urban complexity represents the test case. In a systematic study the quality of the numerical predictions of mean wind fields is evaluated with a focus on the identification of model strengths and limitations. State-of-the-art validation metrics for numerical models were used to quantify the agreement between the data sets. Based on detailed spatial identification of locations of good or bad comparison the study showed how unsteady flow effects within street canyons are a major cause for discrepancies between numerical and experimental results. (C) 2012 Elsevier Ltd. All rights reserved.