Top considerations when building your own network

By Koby Reshef, PacketLight CEO (last updated 19-Apr-22)

Dark fiber, colocation sites, equipment needs, and DWDM integration are important to an organization that will build its own fiber-optic cabling infrastructure.

Buying network infrastructure (or service) from a major carrier may be efficient, convenient, and certainly hands-off, but often comes at a considerable cost. This route is almost inevitably accompanied by higher fees, an inability to economically or dynamically scale, and a reliance on a third party when network problems arise. Now more than ever it is critical for businesses to maintain control over how and when to scale their network infrastructure. Data confidentiality and custom services like Fibre Channel will also become more important considerations moving forward.

For these reasons, organizations are increasingly electing to build their own infrastructure, leasing dark fiber and powering it with owned transport equipment. This is a large undertaking with potentially serious, although surmountable, challenges during implementation. That said, once a foundation is established, self-built infrastructure can dramatically lower capital expenditures (capex) and operating expenses (opex). Within this article are steps IT leaders must consider when building their own infrastructure.

Acquiring dark fiber

First, enterprises must determine their current and future data transport capacity needs, preparing their optical backbone infrastructure to be futureproof. The most common challenges that companies face when building infrastructure are identifying an ideal fiber path and engaging a dark fiber provider that can serve their needs based on monthly cost, fiber path and quality of the fiber. With future growth in mind, it will be critical to make sure the dark fiber provider has paths that match colocation centers in ideal areas in case of expansion. Because capacity per wavelength can grow from 10G/100G/200G to 400G, enterprises should use equipment that can expand these capacities without disturbing existing services or taking up too much rack space.

The last mile continues to challenge those that own infrastructure. Given that a majority of dark fiber terminates prior to the colocation center, last mile infrastructure must be installed between the drop and colocation equipment—a potentially large and hidden expense in both cost and time-to-network. In the United States, sites such as identify dark fiber providers around the country and the routes that their fiber follows, which enables enterprises to select the dark fiber provider that best suits them.

Another factor to consider when selecting a fiber vendor is the number of strands a company will need to purchase, including protection fiber path to protect against fiber cut. In cases of fiber path protection, it is recommended to have two different fiber routes between the sites and, in certain cases, from different fiber providers.

Deciding on a colocation site

The next step is to select a colocation site that best fits the organization’s unique needs (i.e. hybrid cloud, on premise). Colocation sites are facilities leased to businesses to house servers and other hardware, while the colocation site provides services such as power, cooling, and daily management of the data center. Typically, in these facilities, the customers own the equipment and maintain full control over their network’s design and usage.

Factors such as location, power rental costs and security contribute to the ultimate cost of the network—both initially and over time. As an example, for a brokerage firm where every microsecond counts, location is a prime consideration. The proximity of their office to the financial exchange servers is critical in their constant race to reduce latency. It may likely come with higher costs for space and utilities when facilities are located in leading financial capitals such as New York, London, or Beijing. Enterprises less reliant on micro-latency may strongly consider a tradeoff between shortest fiber distance, lower rental and power costs.

For organizations using colocation sites that send/receive highly sensitive information, physical security will be a priority. As an increase or decrease in opex will affect the organization’s bottom line, meticulously weighing these factors will be critical at the onset of the infrastructure buildout.

Defining an organization’s needs for equipment

The next step after selecting a dark fiber provider and a colocation partner is to choose the equipment that will light the fiber. The equipment required will depend on distance (short-haul, metro or long-haul), data throughput, and the data protocols that need to be supported (e.g. Ethernet, Fibre Channel, OTN). Optical transport equipment and wavelength-division multiplexing (WDM) are also important considerations when developing a more-expansive, robust service level agreement (SLA) package with a dark fiber provider. This can increase the value significantly by building a reliable backbone with diverse fiber path and equipment level protection, higher capacities and other ways to maximize the value of the solutions to the enterprise.

Determining the capacity requirements to run the network is the primary consideration in selecting the optical transport equipment. A key question is whether a business will require 10/40/100/200/400G capacity. A company that distributes data to a colocation site across a city may only need short distance optimized DWDM optical network with few intermediate amplification inline sites. A company sending massive amounts of information across the country would likely require higher wavelength spectral efficiency and build 100G, 200G or 400G long-haul DWDM/OTN network.

The next consideration is how the equipment will protect continuous data transit between sites in the WDM layer, of which there are three options. First is full equipment redundancy, which protects against any equipment or fiber cut. Second, facility protection that just protects against the physical layer cut (fiber lines). Third, facility-plus protection, which protects against fiber cut and optical module failure. The tradeoffs are cost and resilience. It’s important to note that nearly 90 percent of the cases are fiber cut rather than equipment failure.

The third consideration when purchasing equipment is the level of security that an enterprise network needs. Data security has become a necessity, rather than a preference. Some industries need security for protection; others have data encryption requirements or demands for compliance. The impending General Data Protection Regulation (GDPR) legislation in the EU will further strengthen the conviction of companies to secure their data in transmission. GDPR is a European data regulation act that places unprecedented data protection levels, heavily fining companies that do not comply, with very little breathing space.

Data-sensitive organizations, like law enforcement, financial institutions, or medical facilities, must strongly consider purchasing single unit equipment with onboard point-to-point physical layer security. This is especially true if the fiber path runs relatively exposed above ground (i.e. train lines). Unfortunately, tapping unencrypted fiber is easy even for unsophisticated attackers, with very simple equipment. It is important to encrypt data without degrading or compromising bandwidth and/or increasing latency. Security of equipment inevitably adds a varying degree of latency across the board, but it’s less at the lower levels, so encrypting data at the physical layer (Layer 1) is more advantageous than Layer 2 or Layer 3 of a network.

The fourth consideration for equipment is scalability. Enterprises must consider future network plans and the scalability of their infrastructure. It is as necessary to chart data and network growth as it is to map the equipment needed to support it. The plan should always seek a solution that avoids the need to replace equipment further down the line. Because network topologies differ, equipment like muxponders and transponders must support multiple options like point-to-point, ring and linear add/drop. Scalability can likewise affect the number of channels a company will need. Top leading providers offer up to 96 channels at 50GHz ITN grid spacing for 100G/200G wavelenghs, or 64 channels at 75GHz ITU grid spacing for 100/200/400G wavelenghs.

Educating on DWDM and integration

An important consideration enterprises must take is the personnel training required. Small to mid-sized organizations seeking to build their own network may not all have the core competency required for fiber-optics networking, or understanding DWDM technology that is now prevalent in many larger organizations. Dense wavelength division multiplexing is simply the means of increasing bandwidth over existing fiber networks. Essentially, it allows transmission of many more signals at different wavelengths within a single carrier wave. More information can be sent at one time over a single fiber than ever before. DWDM helps provide significant improvement to communications and day-to-day workload.

Therefore, network managers and the IT team must be educated about DWDM to ensure smooth integration into the organization’s current infrastructure, futureproofing the network, and minimizing the cost of running the network. Active WDM practices form an independent optical backbone layer, which is agnostic to the Layer-2/3 equipment vendors and type of protocol transported, integrating with existing monitoring tools in Layer 2/3 to make it easier to maintain and manage fault-isolate issues in the optical transport layer.

Building a network has many components, especially during initial implementation. Organizations must take the route of building networks and partner with dark fiber providers with the potential to create flexible networks that can grow alongside their business. 


Published in Cabling Installation & Maintenance
Click here for the original article

Koby Reshef CEO

Koby Reshef has more than 25 years of experience in both the technical and marketing aspects of the telecommunication industry. He brings extensive knowledge of a variety of technologies such as storage networking, optical networking, ASICs, and wireless communication solutions. Before being appointed as CEO, Koby managed PacketLight's HW design team. Prior to joining PacketLight, he served as a senior R&D Manager at Teledata (ADC). Koby has an MBA from Netanya Academic College and a BSc in Computer Science from Tel-Aviv University.