The purpose of this study was to empirically clarify the process of social exclusion, which interferes with health in adulthood, using panel data covering all of Japan. This study focused on the complex problems of people with physical and mental problems. The results suggest that social exclusion in childhood may have a strong impact on adults’ physical and mental problems. There is a need for policy responses to adverse childhood experiences, including social (bullying victimization at school and a negative family atmosphere) and economic (material deprivation) dimensions. Furthermore, the results show that the impact of single points in adulthood differ from cumulative impacts over ten years . That is, this study confirms that there are dimensions that may be overlooked if they are judged only on the situation before a certain point in time. This study is positioned as a life course study to elucidate the impact of the social and economic dimensions of childhood and adulthood on health.