We develop a model where labor market structure affects the division of surplus between firms and workers. Using Austrian data we show that in more concentrated labor markets, workers are more likely to return to past employers. In our model, the possibility of these re-encounters endows firms with size-based market power since outside options are truly outside the firm: firms do not compete with their own vacancies. Hence, a worker’s outside option is worse when bargaining with a larger firm, and wages depend on market structure. The quantified model suggests that such size-based market power could substantially reduce wages.