Hello all! Continuing our story about how the construction of a new data center "Avantage" from scratch, which will be one of the largest commercial sites in the Moscow region. Today we take a detailed walk through the server rooms, go into the cold center, power center, go up to the roof of the data center – it will be interesting. If you missed it, you can read the first part of our story here Attention, lots of photos ahead!
Let’s see what our facility looks like today. And we’ll start with server rooms (hermozones/mashrooms). Total server rooms will be 16, they are located on 2 floors of the technological block, 8 server rooms per floor. Each server room has the size in plan 13, 1 x 30, 2 m (S = 396 sq ft) and height (from clean floor to ceiling) 5, 0 m, capable of holding up to 140 server cabinets 800h1200 mm. The rated power of the server room is 700 kW.
Server room cooling provides precision air conditioners that are installed both inside and outside the server rooms. In 13 out of 16 server rooms, precision air conditioners are installed outside the basement, in dedicated technological corridors. This is done in order to limit and/or exclude the access of facility’s service personnel to the basement and to meet the requirements of the facility’s and/or client’s security service. The air conditioning system of the turbine hall includes 8 air conditioners, with N+1 redundancy, where N equals 7 air conditioners, able to provide 735 kW of heat loss from the technological equipment and maintain the supply air temperature at +24°C. In total, the project envisages installation of 128 precision air conditioners in the server rooms and four air conditioners in two cross rooms.
Let’s take a look at one of our maximum startup-ready hermozzones. The air conditioners for it are located outside in the process aisle (outside air conditioner location) – below in the photo.
Installation is over in the mashrooms trunking and distribution bus ducts which will supply power to customer servers. The main busbar 1600A, supplies power from the main distribution switchboard directly to the distribution busbars in the turbine hall. Distribution busbar rated at 160A supplies power from the main busbar to the client servers. For the connection of IT racks on the distribution busbar installed off-take units with circuit breakers. For each room the power supply is made by a two-beam scheme, from two different main switchboards.
It is worth noting that the distribution system solution is versatile: in case the customer needs to shift rows of IT racks, change aisle dimensions and/or change the width of a row of IT racks, the overhead distribution system can move on the ceiling rail system.
A little bit about security system. The facility is equipped with a video surveillance system, an access control system with customizable mode/level of access for customers and service personnel. A modular video surveillance system is implemented in the halls. Depending on customer needs, this allows the addition of video cameras.
In addition, sound and light fire warning system and any other h/c, aspiration system (early smoke detection system) and, of course, centralized gas fire suppression system (yellow gas fire suppression piping in the photo) are implemented in the turbine halls.
A supply and exhaust ventilation system was also installed to provide overpressure and the required level of humidity in the turbine halls.
Since the air conditioners for this "red" containment area are located outside the hall (we showed them at the beginning), to meet the fire safety requirements in the wall of the turbine hall, on the air supply and air intake of the air conditioner, there are fire dampers (fire dampers, OZK).
Pictured : damper flap actuator in open position (top), air intake and air supply ducts.
Our other containment area is "Yellow" where the air conditioners are mounted directly in the room itself. In this pressurized zone we started loading tests, for this purpose we mounted and connected 7 heat guns to the standard power supply network. Each heat gun – is 100, 000 watts (for comparison: an iron is 1500 watts). The test is for the rated capacity of the mashroom – it is 700 kW. Seven heat guns are capable of providing a heat output of 700 kW. The load tests last 72 hours.
In addition, the creation of internal infrastructure to accommodate customers’ IT capacities is now underway here. In particular, cabinets for client’s servers are being installed. The cabinets are domestically produced by Masterwork-telecom. The cold aisle isolation system is also done by C3 Solutions. By the way, we tested the fire dampers in September and they were great.
There are 7+1 precision air conditioners in the server room, manufactured by Tecnair LV. 7 air conditioners provide the whole hermetic zone hall with cold. Eighth air conditioner is reserve. Each air conditioner provides cooling capacity of 105 kW.
Let’s take a look at another containment area, it’s also "Yellow". The readiness here is lower than the others, no raised floors yet. But we can see the installed gas fire suppression system, which is located not only under the ceiling of the server room, but also under the raised floor. The air conditioners are installed in the technological corridor (in the photo you can see the fire dampers (FVS) installed on the air supply from the air conditioner under the raised floor).
Since the air conditioning of the nearby containment areas is external, the air conditioners have been moved into the process corridor and are located on both sides of it. For the turbine halls of the first start-up complex tests have already been performed (the photo below shows the right-hand row of air conditioners). The air conditioners of the second start-up complex halls (on the left side) are installed, the hydraulic cooling system is also installed and ready for hydraulic testing.
By the way, you can get a good look at sprinklers automatic water fire suppression systems.
Avantage provides for two centralized gas firefighting stations. Together they cover the entire volume of the data center. The stations are absolutely identical, one of which is pictured below. Construction and installation work here was completed in February 2017.
Each station contains 24 gas fire extinguishing modules, main and distribution pipelines, through which the gas extinguishing agent is directed along a distributed route, covering half of all rooms of the data center. Each red module (cylinder) of 180 liters holds 150 kg of extinguishing agent FK-5-1-12 (Novec 1230), by far the safest for people, equipment and the environment and most effective for fire suppression. Modules of gas firefighting system of the second row form a hot standby, which runs in case of failure in the main, first row of cylinders. Manufacturer of this fire safety system is domestic – ASPT Spetsavtomatika.
Fire cabinets with hydrants, hoses, and fire extinguishers are located throughout the corridors of the data center.
Refrigeration Center – is the heart of the air conditioning system of the data center server rooms. It is organized in a separate one-story block. At the moment cooling machines (chillers) are mounted there, starting-up and adjustment works of the first start-up complex are fulfilled, cooling system is tested with a test heat load to the designed capacity. At the same time the installation works of the second start-up complex are in progress.
Refrigeration machines (chillers) imported, manufactured by Cofely (Germany). Indoor chillers with liquid cooled condenser with Turbocore centrifugal compressors. On photo below is one of the refrigeration circuits and chillers themselves.
Air ducts of the cold center process ventilation system.
Each chiller has its own power supply switchboard. Each chiller is provided with two independent power feeds. The power takeoff box of the chiller power supply panel is visible on the bus duct.
Now let’s talk a little bit about power supply system "Avantage. In case of a blackout, the data center has an uninterruptible and guaranteed power supply system (UPSS). It is organized on the basis of diesel rotary uninterruptible power supplies (DRIBP) , produced by Hitec Power Protection (Holland), mounted on the 1st floor of the 2-storey block of the power center. DRIBP were implemented in the power supply system at the 10 kV level. A total of 14 units will be installed at the site in two stages. The level of redundancy SBGE N+1. In the first phase, 5 units are put into operation: 4 main (N), the fifth redundant (+1). In the second stage all 14 units: 13 main (N), fourteen standby (+1).
Pre-commissioning work on the first phase DRIBPs was completed in November. The units operate continuously, providing filtration of electrical energy from the grid to the consumer and from the consumer to the grid.
In case of loss of external power supply, the facility is provided with fuel reserve for DRIBP, which ensures autonomous operation of the facility for a period of not less than 12 hours at 100% load of the facility. All engine/diesel rooms are equipped with a gas fire suppression system from the same centralized stations as we have shown above.
Opposite the DRIBP rooms are halls with boosting transformers The site is located opposite the DRIBP rooms, where the voltage level is increased from 0.4 kV from the DRIBPs (the DRIBPs themselves are designed for 0.4 kV) to 10 kV in the distribution network of the power center. The facility is provided and installed 14 power step-up transformers 0, 4/10 kV, manufactured by Siemens.
Power distribution for medium voltage 10 kV was built on the basis of medium-voltage cells manufactured by Siemens. They were installed on the 1st floor of the power center block. This voltage level provides reception of power supply from the city grid, connection of DRIBP and distribution to two independent parallel bus sections of the uninterruptible and guaranteed power supply system, connection of step-down transformers of consumers.A total of 11 medium-voltage sections are provided and installed by the project.
Today the power supply system for medium voltage (10kV) and low voltage (0, 4kV) is in pilot operation. We have 6 such premises with distribution cells, 112 cells 10 kV of NXair, Simosec and 8DJH series.
In the photo below is the control cabinet of the generating sets of the parallel system. From here the visual control of the operation of the DRIBP machines is carried out. Next to it, across the aisle, there is a DC cabinet with a set of batteries, which provides independent power for control circuits of medium-voltage cells and control panels of DRIBP units.
27 10/0.4 kV step-down power transformers manufactured by Siemens were provided and installed at the site.
The second floor of the power center block houses the process ventilation system and the control system for the diesel-rotor uninterruptible power supplies.
Let’s go up to the roof and have a little look at the data center from above. Here above the technology block rooms are 18 dry coolers manufactured by Cabero. In the project, dry coolers are used to discharge excess heat into the atmosphere. At outside air temperatures below +6°C the data centre cooling system switches over to direct cooling mode (Freecooling), which does not use cooling machines (chillers).
Here in the photo you can clearly see lightning masts. The facility is protected by a lightning protection system. Equipment placed on the roof is protected from a direct lightning strike by a combination of lightning protection grid and lightning arrestor masts. Inside the facility, surge protectors with surge arresters are used on the input lines from the city network and on the cable lines of the equipment that is placed outdoors.
The photo below shows the exhaust pipes from the generating sets of the power plant and its ventilation system (intake and exhaust shafts).
A word or two about fuel storage Its volume is 180 m³, designed for 24 hours of autonomous operation at full load. In the photo you can see the necks of the fuel tanks for diesel fuel buried in the ground. Drainage from the fuel tanker into the tanks takes place through the refueling station.
The first telecommunications facility appeared at the Avantage construction site back in March 2017. Cellular operator MTS installed a mobile base station at the construction site of the data center. Thus, there is good mobile coverage in all rooms of the facility under construction.
At the Avantage site, a physical security loop has been closed, meeting the highest requirements. The work was completed back in July 2017. The reasons for choosing such protection :
- rigid welded construction;
- minimal possibility of intrusion;
- anti-tamper device suppresses unauthorized intrusion by intruders under the main security fence.
The next step was to install video cameras and vibration sensors.
Beauty has a meaning
Well, of course, a year ago we weren’t even talking about beauty yet. For comparison with KDPV, here are a few pictures of what our data center looked like a year ago. The buildings are already standing, but we haven’t even heard anything about the landscaping.
We started landscaping back in the winter of 2016, but we didn’t get much done before the snowfall. So as soon as the snow came off in the spring of 2017, we got to work on the area. What has changed today? Now we have smooth roads, green lawns (in summer, of course)… well, everything is as it should be.
The data center building itself is now a sight to behold. On the end of the administrative building is a huge logo with the name – Avantage Data Center.
In May 2017, work on the architectural lighting of the data center began. And gradually it became beautiful both during the day and at night. By September, the outdoor and architectural lighting subsystems were fully completed at the site.
That’s all for today. If you have any questions, please feel free to ask in the comments. And anyone interested in seeing it in person is welcome to take a free tour. You can sign up here