Dear,
The GBTCLK_M2C is the transceiver reference clock used by the carrier board MGTs. The FMC176 is equipped with the AD9517 PLL which generates the clocks for the ADC/DACs and the GBTCLK (refer to the User Manual, figure 6). In the standard reference app, this clock is configured to have 1/2 the frequency of the ADC Sample clock. When configuring the transceiver inside your FPGA, you should set this frequency to the same value to make sure the transceiver will lock to the serialized data stream received from the ADC. If you want to change your ADC sample rate you have to make sure that the new reference (GBTCLK_M2C) clock, provided to the MGT is within the frequency range of the receiver PLL.
Best regards,
Ingmar van Klink