Modeling experts have been continually researching the interplay of human mobility and COVID-19 transmission since the outbreak of the pandemic. They tried to address this problem and support the control of the pandemic spreading at the national or regional levels. However, these modeling approaches had little success in producing empirically verifiable results at the neighborhood level due to a lack of data and limited representation of low spatial scales in the models. To fill this gap, this research aims to present an agent-based model to simulate human mobility choices in the context of COVID-19, based on social activities of individuals in the neighborhood. We apply the VIABLE model to the decision-making process of heterogeneous agents, who populate the system's environment. The agents adapt their mobility and activities autonomously at each iteration to improve their well-being and respond to exposure risks. The study reveals significant temporal variations in mobility choices between the groups of agents with different vulnerability levels under the Covid-19 pandemic. Agents from the same group with similar economic backgrounds tend to select the same mobility patterns and activities leading to segregation at this low scale. We calibrated the model with a focus on Porto Alegre in Brazil.