Muwatta Malik 1276

وَحَدَّثَنِي عَنْ مَالِكٍ، عَنِ ابْنِ شِهَابٍ، عَنْ عَمْرِو بْنِ الشَّرِيدِ، أَنَّ عَبْدَ اللَّهِ بْنَ عَبَّاسٍ، سُئِلَ عَنْ رَجُلٍ، كَانَتْ لَهُ امْرَأَتَانِ فَأَرْضَعَتْ إِحْدَاهُمَا غُلاَمًا وَأَرْضَعَتِ الأُخْرَى جَارِيَةً فَقِيلَ لَهُ هَلْ يَتَزَوَّجُ الْغُلاَمُ الْجَارِيَةَ فَقَالَ لاَ اللِّقَاحُ وَاحِدٌ ‏.‏

Yahya related to me from Malik from Ibn Shihab from Amr ibn ash- Sharid that Abdullah ibn Abbas was asked whether, if a man had two wives, and one of them nursed a slave-boy, and the other had nursed a slave-girl, could the slave-boy marry the slave-girl. He said, "No. The husband is the same