Effectively reducing the voltage loss in organic solar cells (OSCs) is critical to improving the power conversion efficiency (PCE) of OSCs. In this study, highly efficient ternary OSCs were constructed by adding a non-fullerene acceptor Qx2 with a high open-circuit voltage (VOC) and low energy loss (Eloss) into PM6:m-BTP-PhC6 based binary devices. The third component Qx2 shows slightly complementary absorption with m-BTP-PhC6 and also optimizes the molecular packing, orientation, and morphology of the active layer. Moreover, the incorporation of Qx2 reduced the energetic disorder and improved the electroluminescence quantum efficiency, which suppresses the Eloss and further leads to a higher VOC than the PM6:m-BTP-PhC6 binary blend. Consequently, synergetic enhancements of VOC, short circuit current (JSC), and fill factor (FF) are realized, resulting in the PCE of 18.60%. This work shows that the selection of the appropriate third component has positive implications for reducing Eloss and improving the PCE of OSCs.