Foreign currency is held by the central bank for two primary purposes
To pay off any external debt. For e.g India hold US dollar to pay up the loan taken by India from foreign banks such as world bank.
To maintain the domestic exchange rate. For e.g. If rupee is depreciating against US dollar then the central bank (RBI) will sell dollar from its reserve. It will cause the US dollar to depreciate as the demand of the US dollar is reduced which in turn will cause the Rupee to appreciate