The Yolk and White are very different and I would expect to see a significant difference in result.
That isn't to say it will be bad, just different.
The Yolk is the source of fat. It is going to impart flavor to the dish and provide a creamier texture.
The White is mostly protein and so will set more, have more of a texture impact.
If it called for several whole eggs and you wanted to take a Yolk or two out for less fat, that would have less of a variance, but switching several Yolks to several Whites should definitely alter the results.
Here is some nutritional information of a Large Whole/White/Yolk.
You will see they both carry protein however the Yolk has almost twice as much fat as protein while the White has less than 2% of fat as it does protein.
This makes their behaviors very different.