Fudge is candy. Like all candy making it is built on a concentrated sugar syrup. Fudge is differentiated from other candy in that it is encouraged to form tiny crystals and is high in fat.
The chocolate in fudge provides two things: flavor and fat. Although less traditional, plenty of recipes for fudge variants leave out the chocolate altogether.
As @ElmerCat said above, chocolate candy already has a ton of sugar in it. This wouldn't be a problem but it might change the timing of your candy cooking. More sugar to liquid means the candy will not need to cook as long. You'd have to watch your temperature.
The real problem is that chocolate candy like that doesn't taste much like chocolate at all. You would be able to make fudge with the right texture and color but it would taste bland.