Bimodal morphologies of massive galaxies at the core of a protocluster at z=3.09 and the strong size growth of a brightest cluster galaxy

Abstract: We present the near-infrared high resolution imaging of an extremely dense group of galaxies at the core of the protocluster at $z=3.09$ in the SSA22 field by using the adaptive optics AO188 and the Infrared Camera and Spectrograph (IRCS) on Subaru Telescope. Wide morphological variety of them suggests their on-going dramatic evolutions. One of the two quiescent galaxies (QGs), the most massive one in the group, is a compact elliptical with an effective radius $r_{e} = 1.37\pm0.75$ kpc. It supports the two-phase formation scenario of giant ellipticals today that a massive compact elliptical is formed at once and evolves in the size and stellar mass by series of mergers. Since this object is a plausible progenitor of a brightest cluster galaxy (BCG) of one of the most massive clusters today, it requires strong size ($\ga10$) and stellar mass ($\sim$ four times by $z=0$) growths. Another QG hosts an AGN(s) and is fitted with a model composed from an nuclear component and S\'ersic model. It shows spatially extended [O{\footnotesize III}]$\lambda$5007 emission line compared to the continuum emission, a plausible evidence of outflows. Massive star forming galaxies (SFGs) in the group are two to three times larger than the field SFGs at similar redshift. Although we obtained the $K$-band image deeper than the previous one, we found no candidate new members. This implies a physical deficiency of low mass galaxies with stellar mass $M_{\star}\la4\times10^{10}~M_{\odot}$ and/or poor detection completeness of them owing to their diffuse morphologies.