And he was altogether wonderful in faith and religious, for he never held communion with the Meletian schismatics, knowing their wickedness and apostacy from the beginning; nor had he friendly dealings with the Manichæans or any other heretics; or, if he had, only as far as advice that they should change to piety. For he thought and asserted that intercourse with these was harmful and destructive to the soul. In the same manner also he loathed the heresy of the Arians, and exhorted all neither to approach them nor to hold their erroneous belief. [Life of Anthony 68]
All we have to do is remember that Athanasius was the author of this pseudo-history. The very fact that he references communions with the Manichaeans is extremely important. Anthony's association with the heretics must have been well known. Athanasius and Anthony might have shared mutual hatred of the Arians.
Stephan Huller has had a life long interest in religion and spirituality 
dating back to conversations with his grandfather, Gaston Frank. "He said we represent one of the last descendants of the Frankist Jewish faith in the world," he muses. "I grew up thinking that our family was something like the Last of the Mohicans."