With the increasing emphasis on reducing the CO2 emission while improving power generation efficiency, new power cycles are being developed. One of those promising power cycles is a supercritical CO2 (S-CO2) power cycle. To generate over 10MW electricity with S-CO2 power cycle, a magnetic bearing can be a good option for the hermetic type turbomachinery. However, from several studies on the magnetic bearing, the instability issues under high density fluid and high speed operating conditions were repeatedly mentioned. The instability in the magnetic bearing was observed to be related to the fluid conditions, mostly pressure and density. Because of this issue, the magnetic bearing sometimes cannot maintain enough clearance for the rotor leading to physical contact and consequently damaging the system. Thus, these instability issues should be thoroughly studied and be resolved for the successful and steady operation of the power system. The instability due to fluid force around the rotating shaft can be modeled with the Reynolds lubrication equation. The steady lubrication force analysis model is developed based on this equation. The model results imply that the lubrication performance is quite sensitive to the thermal condition of the CO2 especially density gradient around the shaft. Based on the modeling results, an experimental system is designed to investigate the issue. To study the instability issues experimentally, an impeller of the operating S-CO2 compressor is removed and the discharge line is blocked. Therefore, the main instability factor in this experiment will be the interaction between the rotor and the bearing only. The shaft position can be measured with inductive sensors. The forces exerted from the electromagnet is calculated from the electric current data which is applied by the controller. From these experimental data, the lubrication force is calculated. These results are compared with the analytical lubrication model to verify the model. From this study, it is expected that it will be possible to define the unstable operating conditions and suggest the required magnetic bearing performance for S-CO2 conditions.