This paper examines how James Joyce discloses the essence of masculinity through female characters’ bodies. Joyce’s position in feminist criticism has been polarized, especially regarding his description of masculinity. While some feminist critics have argued that Joyce is one of the exemplary patriarchal artists in the twentieth century, others espouse him because of Joyce’s use of femininity as an expression of otherness. This article synthesizes both of them, arguing that Joyce’s misogynic charges stem from his meticulous description of the difference between man and woman. By portraying Stephen’s negation of his own body and women’s reduction to his idea, Joyce describes how masculinity as an idealism attempts to reconstruct and recreate women in accordance with the phallic fantasy. Moreover, Bloom’s perception of women’s partial body also signifies that in his mind there is one form of the body that stimulates Bloom’s sexual pleasure. However, it is the female body that delays and impedes their phallic fantasy. Using Maurice Merleau-Ponty’s conception of the body, this paper finally maintains that Joyce depicts women characters’ flights from masculine signification through their bodies.