OSLO—Russian oil giant OAO Rosneft has agreed to buy a large stake in a New York-listed oil-services firm—deepening its relationship with a company that has honed Arctic-drilling capabilities just as the West tries to restrict Moscow’s access to such technology through sanctions.

North Atlantic Drilling Ltd. said Friday that Rosneft would acquire a 30% stake in the company, in exchange for an undisclosed amount of cash and 150 of…