Without Toshio there to break the ice, Naoki and Kouki didn't get to know each other very well. You can say Naoki left early, tired of sushi and awkward silence, while Kouki waited for Toshio but left when the bill arrived. When they see each other again, they will recognize each other, but remember their awkward "lunch date".