Beginner’s Web Development Guide

Foreword

Web development is currently one of the most developing and popular IT sectors. Users are craving richer and more powerful applications and new emerging technologies increase developer abilities to satisfy this demand. We see new types of services, media platforms and cross-platform applications for mobile and desktop built using web technologies. As a result, the demand for web developers is still much higher than the supply and it's a good area to start your development career in.

For beginners, it’s easy to get into web development. There are many languages with relatively low entry barriers, a lot of free open-source tooling and many frameworks that simplify development. On the other hand, there is so much to choose from. The difficult part about development is that you need to combine a lot of components together for your application to work. Which UI framework to choose? Where to store your data? PHP, Python, Ruby or something else? How to host your application? This guide will shed some light on this and help you figure out your way around web development and maybe even start a career.

The guide is not meant to be a complete learning material, but rather an overview of the industry. Each chapter provides a descriptive list of the major topics in this area and links to in-depth learning materials. To make it easier to follow the topics, each chapter starts with a mind-map to give a visual overview of the topics covered. We try to keep the list as objective as possible by taking into account data on usages of a particular technology as well as developer surveys, however, we also reflect our own personal experience. When following the guide, we suggest you familiarise yourself with the basic topics before continuing the more advanced ones.

Contents

The guide is split into four major chapters.

1. Frontend Development

This section will introduce you to the basics of modern UI development, including the most popular languages, tools, and frameworks.

2. Backend development

In chapter 2 we'll go through the different languages used for server-side development as well as have a look at different databases and messaging techonologies.

3. Platforms and Tools

Chapter 3 will give an overview of some fundamental technologies of web development, such as networking, web servers, virtualization as well as the most popular development tools.

4. Cloud Deployment

The final chapter will focus on deploying the application in the cloud and the different options and services available on the market.

Conventions Used in the Guide

Colors are used in the chart to indicate the level of complexity of each topic:

  • yellow lines show the basic things you need to learn;
  • orange - represent more advanced topics that you can leave for later;
  • grey - are niche topics that you might come in handy on some occasions.

Frontend

Frontend development, also known as UI development, refers to creating web user interfaces, the parts of the application that the user sees and interacts with. Frontend development stands firmly on three pillars: HTML for markup, CSS for styling and JavaScript for logic and interactions. Although these three technologies stood the test of time, the underlying tools and frameworks change constantly. This section will give you a starting point to start the exploration of the frontend ecosystem.

Frontend development is also tightly related to web design and user experience and the border between the two can be different in each company. In general frontend developers focus on technical issues, while web designers are concerned with aesthetics and user experience.

Frontend map

HTML

Hyper Text Markup Language (HTML) is a markup language that allows you to structure the information that is displayed to the user. Its the An HTML document consists of elements each represented by an HTML tag, such as a heading, a paragraph of text or a text input field. HTML documents can also link to other documents using hyperlinks. Each tag carries its own type of content, has a certain appearance and attached behavior. Once you get a hang of HTML, you will be able to create very simple static HTML pages.

CSS

Cascading Style Sheets (CSS) is a styling language used to describe the visual appearance of HTML elements. It can be used to controls aspects like the positioning of an element, its dimensions, text style or colors, or almost any other visual aspect of a website. Styles are defined as a set of rules that can be applied to a single element of a subset of elements on the page. Recent CSS features allow you to go way beyond basic styling and control more complex visual aspects, such as animations.

Preprocessors

CSS is a powerful language but it has its constraints when it comes to code reuse, verbosity, and isolation. Because of this, maintaining a large CSS code base can be difficult. To address these shortcomings, a number of solutions have been created by the community. These languages and their compilers are generally referred to as CSS preprocessors.

  • Sass/SCSS extends the capabilities of CSS to make your CSS code base more maintainable. Some of the features, such as nesting, make CSS less verbose and easier to write, while other such as variable and mixins allow for code reuse. It also supports importing other SCSS files that allows you to break your code into smaller files. Some popular CSS frameworks such as Compass are based on Sass.
  • PostCSS is a tool that uses JavaScript plugins to transform CSS files. You can add plugins that perform code analysis, add variable support, optimize code and etc. In contrast to SCSS or Less, which are custom formats, there are a lot of plugins for PostCSS that emulate the upcoming standard CSS features.
  • Less is a preprocessor with a feature set similar to Sass. Despite their similarity, there are a number of differences in the underlying implementations. Although being popular in the past, a lot of developers are now moving away from Less in favor of Sass or PostCSS.

JavaScript

JavaScript (JS) is the most used programming language for developing user interfaces for the web. It allows you to add custom behavior to your websites, such as reacting to user events, loading data from web services and showing or hiding elements on a page. It also allows you to interact with the browser through various APIs. You can use it to both add separate interactive elements to your website or create complex web applications.

Another term that you may encounter related to JavaScript is ECMAScript which is actually a language specification for JavaScript. In most cases, they can be treated as synonyms.

Frameworks

JavaScript is a powerful language, but if your looking to make a complex application, using it without a framework will result in masses of complex boilerplate code. To make it simpler, there are a number of frameworks out there that help with the basic functionality such as HTML manipulation, data loading, overall application architecture and etc. The three most popular frameworks at the moment are React, Angular and Vue.js.

  • React is a library for building user interfaces developed by Facebook. React is based around creating components, such as a button or a side menu, that can render HTML based on input parameters. React components are written using an extension to JavaScript called JSX, that allows you to conveniently combine JavaScript code and HTML. Components can be composed together to create complex UI elements and whole applications. React is not a framework per se, it only provides the view layer, however, its rich ecosystem of tools and libraries has everything you’ll need to create complex applications. React is easy to start with, but with time you are expected to learn new libraries to fill the missing gaps.
  • Angular — a component-oriented “batteries included” type of framework developed by Google. Angular applications are usually written in TypeScript. It provides most o the things you’ll need to build a web application including a CLI tool to bootstrap, develop and build the project as well as libraries to manage forms, data loading, routing, etc. Although it does provide most of the things out of the box, it has a higher abstraction and complexity levels, and the need to learn TypeScript as a primary development language makes the initial learning curve steeper.
  • Vue.js is a framework designed to be modular and incrementally adoptable. You can start using it as a view layer and add additional libraries as you need them. In contrast to other frameworks, the most important libraries are maintained by the core Vue.js team.

You might also encounter other frameworks out there that have a smaller community or are getting superseded, such as Ember.js or AngularJS.

TypeScript

TypeScript extends JavaScript to add static type checking and the latest ECMAScript features, such as async/await, classes or arrow functions. As a result, TypeScript compiles into vanilla JavaScript. Having static type checking in place means that the compiler will notify you when you are trying to use a function or variable in a manner that does not match its signature. This also gives code editors the ability to provide better code navigation and refactoring capabilities. Typings are optional, so you can introduce them gradually in your existing JavaScript project. Using TypeScript makes it much easier to work on projects with larger code bases.

Component Libraries

Component libraries are collections of components that can be used to assemble your application. Some of them implement the most common components such as buttons and drop downs, while others also provide utilities for creating complex responsive layouts. Although each of them has its own distinctive visual style, they can usually be customized to match the design and branding of your project. Most of them are available as plain CSS and JavaScript, however, they are often ported to work with other popular JavaScript frameworks. Some of the more popular libraries are:

  • Bootstrap is a powerful framework for creating responsive web applications. It comes with a large set of components as well as utilities for creating layouts and a theming engine. You can add user interactions to components, such as a dropdown, minimal knowledge of JavaScript. There is also a wide selection of ready-made website themes built with Bootstrap.
  • Semantic UI is another popular UI library with all kinds of components and a theming engine. However, to make the components interactive, you are expected to write some JavaScript code.
  • Material UI — a popular implementation of Google’s Material Design. It provides a set of components as well as guidelines on how your application should look and feel like according to Google. There are other implementations available for Angular or vanilla JavaScript.

Some other honorable mentions include ant-design, Foundation, Bulma and Pure.

Frontend tools

The frontend ecosystem is known for its large arsenal of helpful tools each designed for their own purpose. Since there are so many of them, there’s no way we can cover everything in this guide, so instead, we’ll focus on several that are important to get started with.

NPM

Npm actually means two things:

  1. A software registry that contains thousands of open-source libraries and tools that you can install and use in your project.
  2. A CLI tool that allows to install, update and remove packages from the registry.

Using npm allows you to install libraries that will be used by your project and document them in a JSON file. Storing this file together with the code base means that you can explicitly declare the dependencies of your project and quickly install them on different environments. You can also use it to publish your own libraries and share them with other developers.

The npm CLI tool also allows you to run scripts that can be used to execute tests, compile code or perform code analysis.

An analog to npm is Yarn, that offers some performance improvements and better support for library development and is compatible with the npm package registry.

Webpack

Webpack is a popular module bundler. It allows you to merge different kinds of assets, such as JavaScript code, CSS styles, HTML templates and even images into bundles that can be more effectively loaded by the browser. When using Webpack, you split your code into modules. Each module can import other modules. Webpack then analyses the import graph and bundles resources together. Apart from bundling, Webpack uses loaders and plugins to transform bundled code, e.g. perform compilation, code analysis or minification. Webpack can be tricky to start with, but it’s the cornerstone of most of the modern build setups.

Rollup.js is sometimes used as an alternative to Webpack.

Chrome DevTools

Chrome DevTools is a rich set of tools available in the Chrome browser. It can be used for a wide range of debugging cases, such as browsing the source code or HTML elements, tracking network requests and measuring the performance of different parts of your application. It allows you to edit pages on the fly which can be useful to quickly test a small set of changes. Having a firm grasp of this tools will save you countless hours of debugging. You can also use it to look under the hood of other web pages and see how they work. Similar tools are also available in other browsers.

Backend

Backend development, also sometimes called server-side development, refers to developing the processes that happen on the server and are usually not visible to the users. A web UI can send requests to the backend asking to render some HTML, save user input in a database, authenticate a user, send an email or process a payment request. Although this part of the application is transparent for the user, it’s in no way less important. This is the heart and brains of your application.

Map of the backend development landscape

Languages

Compared to frontend development, the choice of programming languages on the backend is much wider and you are free to one a solution depending on your use case and environment.

There are many ways to classify programming languages, and we made an arbitrary decision to group them by scripting and compiled languages. Have to admit, though, that > this is not completely accurate. Some of the scripting languages are actually compiled on the fly, and some compiled languages could have interpreters. This division, > however, serves as a convenient starting point for our analysis.

Scripting languages

Languages that don’t require explicit compilation before execution are called scripting languages, or interpreted languages. As opposed to compiled languages, they do not produce a binary executable. Rather the code is evaluated and run on the fly. Sometimes, scripting languages are easier to get started with, since you don’t have to write code in a strict manner required by the compiler. This, however, means that errors that could have been discovered during compilation will only become apparent during runtime.

Some of the more popular scripting languages are:

  • JavaScript using the Node.js server-side runtime. It supports its own set of JavaScript features and provides a set of API’s to satisfy backend use cases, such as working with the file system. Learning Node.js is a great choice since by learning a single language you’ll be able to work both on the frontend and the backend. Node.js has several popular frameworks, such as express, koa or LoopBack.
  • Python is a general-purpose programming language. It’s simple syntax and clear constructs make it quite easy to learn for beginners. Python is widely used for developing applications but has also gained significant traction for doing scientific research and working with artificial intelligence. There are currently two major versions being used: Python 2 and Python 3. One of the popular projects developed on Python is the Django content management system. Flask is a popular micro-framework for web development.
  • PHP — another popular programming language for web applications. The language itself has a low entry barrier for beginners but also provides more advanced features for intermediate users. Some of the most popular projects created on PHP are WordPress and Drupal, with the former being arguably the most used content management system in the world. Notable frameworks are Laravel, Symfony and CodeIgniter.
  • Ruby — a scripting language designed to be simple and natural to read and write. The most popular development framework for Ruby is Ruby on Rails. Jekyll, a popular static website generator is built on Ruby.

Compiled languages

Code written using a compiled language needs to be first compiled into a binary file before it can be executed. Although this needs an explicit extra step to be performed by the developer, it also brings additional benefits. The compiler can analyze the code for mismatches and give you early feedback about potential problems. Some compilers can also optimize the resulting bytecode for a particular platform ensuring superior performance. Although compiled languages might have a higher entry barrier than scripting languages, they are usually preferable in large projects due to the additional safety measures and more descriptive code.

There’s a large selection of languages to choose from.

  • Java is a high-level object-oriented programming language developed by Sun Microsystems and currently maintained by Oracle. Java designed to be portable between different platforms: the sources are compiled to Java bytecode that can be run on a Java Virtual Machine for any platform. It’s a highly mature language with a lot of different frameworks for developing web applications, the most popular being Spring. Additionally, Java can be used for developing Android applications. Although there are trends in considering Java old fashioned and inferior to other JVM languages, such as Kotlin, it is still one of the most popular programming languages. Java is extremely popular in the enterprise sector.
  • C# is another high-level object-oriented programming language developed by Microsoft. Originally developed to be used on Windows, it can now be used to build cross-platform applications using the relatively new .NET Core runtime. One of the most popular frameworks for building web applications using C# is ASP.NET Core. C# is popular among companies adopting Microsoft’s technology stack.
  • Go is a relatively young language developed by Google that’s quickly gaining popularity. It’s designed to be less verbose than Java or C# and based on simple primitives, intentionally leaving out some of the complex constructs that are present in other languages. Go is extremely fast, has a minimal memory footprint and has first-class concurrency support, so it’s a good candidate for projects where speed and low resource consumption are of importance. The library and tool ecosystem around Go is still developing and we’re seeing a lot of interesting projects appear among the community. Go has gained significant popularity in blockchain development.
  • Kotlin is a language developed by JetBrains and a close competitor of Java. Kontlin runs on the Java virtual machine, but can also be compiled into JavaScript. It’s designed to be less verbose than Java, adds first-class function support and offers additional safety measures to avoid common errors. Like Java, it can also be used to develop Android applications.
  • Scala is another JVM-based language and an alternative to Java. It has a more concise syntax, additional safety features and makes heavy use of functional programming.
  • Erlang — a functional language designed for concurrency and high availability. Erlang is popular in applications where high performance and scalability is critical. Popular use cases include image and signal processing or analysis of large amounts of data.
  • Haskell — another functional programming language that is valued for its powerful typing system and a design that helps to produce clear and reliable code. Since it’s a functional language, it also scales efficiently and offers good performance.

Databases

An important ability of most applications is to store user’s data. It is usually done using special software called database management systems (DBS). A DBS provides access to underlying data storages allowing allow the applications to store, retrieve and update data. There are different kinds of databases optimized for different shapes and volumes of data. In this guide, we’ll have a look at the two most popular types: relational and non-relational.

Relational

Relational databases store data in tables, where rows represent entries, and columns — entry attributes. Rows in different tables can be linked using foreign keys to represent relations between different types of entries. For example, if you’re building an online store, you might have a table for storing orders with columns such as “order number”, “client name” and “shipping address” where each row represents a separate order. Information about the ordered items will be stored in a different table called “order items”, with references to the order table.

Most relational databases use a language called Structured Query Language (SQL) or its dialect to query or modify the data. Because of this, they are often called SQL databases.

Popular relational databases are:

  • Oracle Database is a database developed by Oracle. Due to its high scalability and powerful feature set, it’s often used by large enterprises. Its less frequently used for small and midsize projects due to its high license costs.
  • MySQL — a free open-source database developed by a Swedish company MySQL AB and currently owned by Oracle. MySQL can use different storage engines for each table to store and use data differently. MySQL is used for a wide range of cases, however, it is very popular for web applications, especially in the so-called LAMP (Linux, Apache, MySQL, PHP) stack. It also offers several paid editions for large-scale users.
  • SQL Server also referred to as MSSQL, is a database maintained by Microsoft. It has a number of different editions for handling various workloads and data volumes. SQL Server is a major competitor of Oracle and is popular among Microsoft stack users.
  • PostgreSQL — is a free open source database developed by a worldwide team of volunteers. It provides a rich set of features and is capable of handling large workloads.

Non-relational

Non-relation databases store data in models other than tables. There are different models you can choose depending on your use cases, such as storing key-value pairs, documents, graph relations or time-series. Although non-relation databases have been around for a while, they have recently experienced a surge of popularity due to simpler design and easier scaling.

Non-relation databases are also referred to as NoSQL databases since most of them use their own query language instead of SQL.

These three databases are frequently used in web development.

  • MongoDB is a free open source document database that stores JSON-like objects. You can store and retrieve these objects and map them to objects in your application. It also has the usual querying and aggregation capabilities. MongoDB is a good candidate for applications where data consistency is not a primary concern.
  • Redis is an in-memory key-value database often used to temporarily store cached data and transfer messages. Since such data does not necessarily need to be persisted, saving data to disk can either be disabled or delayed, to increase performance. Redis can store different values types, such as strings, numbers, lists, maps etc.
  • Elastic Search is an open-source text search and analytics engine. It allows you to store text documents and produces a searchable text index. Elastic Search is commonly used for log analytics (particularly as part of the ELK stack) and free text document search.

Messaging

Sometimes server-side components need to trigger jobs that need to be executed in the background. For example, asking for a business-reports may trigger a lengthy process that will generate the report and e-mail it to the user, without the user having to wait for it. To do that we need to be able to send a message to a different component and be sure that it’s processed successfully. If the job fails, it needs to be restarted. Software that provides such functionality is called a message broker.

The two most popular messages brokers are:

  • RabbitMQ is a traditional message broker that supports different message delivery models and routing rules. It guarantees that the message will be delivered and processed by the consumer. Failed messages can be stored in the dead-letter queue. RabbitMQ works through the standard AMPQ protocol that is also supported by other message brokers and client libraries.
  • Apache Kafka is a stream processing engines, that can also be used as a message broker. Apart from providing most of the standard message broker features, it also offers advanced features, such as being able to re-process messages from history. It also provides greater performance that allows processing high volumes of messages.

Platforms and Tools

This chapter covers several large topics:

  • The basic development tools that every developer needs to be familiar with.
  • Since our applications live on the Web, we’ll go through some fundamental topics related to networking;
  • The environment where our applications are run.

Map of development platforms and tools

Development tools

From early on to make sure you are comfortable with your development tools so that you can quickly progress in writing code and debugging your application. Every platform you develop for will have a lot of tools particular for that platform, however, there are some that are used by almost all developers. Let’s walk through them.

Code Editors

The code editor is the home of every software developer. This is where the magic happens. Basic code editors provide the most necessary feature set, such as code highlighting, refactoring and debugging capabilities. There are also more advanced editors, known as integrated development environments (IDEs), that have a wide range of powerful integrated tools and capabilities. Here are some of the most popular code editors.

  • Visual Studio Code is a free cross-platform code editor created by Microsoft. It has built-in support for JavaScript, TypeScript, CSS, and HTML, an support for a lot of other languages is available as extensions. VS Code offers syntax highlighting, code completion, code snippets for generating code, refactoring capabilities, and a built-in terminal. It also has an integrated debugger to help you troubleshoot bugs in your application. Although the feature set might not be as wide as in commercial IDEs, it’s certainly enough to get you started.
  • JetBrains IDEs — a family of IDEs developed by JetBrains for different platforms. There’s WebStorm for frontend development, [Intellij IDEA](Intellij IDEA) for Java and PhpStorm for PHP and etc. Each of these focus on supporting the corresponding languages, frameworks and tools. Compliment this with even more helpful features, such as integration with version control systems, local change history or running tasks, and you will get a highly efficient working environment. These products are paid for, but the IntelliJ Community Edition can be used for free.
  • Visual Studio IDE — is a commercial IDE developed by Microsoft. It comes in three offerings: the free community edition, professional and enterprise. It has support for a wide range of programming languages and different tools for developing and debugging your applications. On top of that, it provides helpful integrations with other Microsoft products and services such as Azure or Azure DevOps. C# and TypeScript are first-class citizens in VS.

Version Control Systems

Version control systems (VCS) play a key role in every development process. VCS allows you to store a complete history of all your code changes in a remote repository. You can browse the repository to review the change history, compare different versions or rollback to an older version. Multiple developers can push changes to the same repository to collaborate on a project keeping their local codebases in sync. Nowadays VCS are used as the ultimate source of truth for the codebase and are a backbone of modern development processes.

VCS systems can be used via a command line interface, but there are also different GUI clients available as well as integrations with different code editors.

There are two popular version control systems:

  • Git is by far the most popular VCS. It’s a distributed system, meaning that when a user copies the repository, he or she also gets the full history of the project and can work with it offline. Occasionally users need to publish their changes to the central remote repository and download changes made by others. There are many services proving hosted Git repositories. The most popular of them are GitHub and BitBucket.
  • Subversion (SVN) is another version control system. Unlike Git, SVN allows you to check out only a single branch. To switch to a different branch, you’ll need to download it from the remote repository. Although still used in the industry, Git is much more popular and has much more tools and services built around it.

Terminal or Command Prompt

As a developer, it’s important that you feel comfortable with using the terminal on OS X or Linux or the command prompt in Windows. Even if you can do most of your work from the GUI, you will still encounter a lot of useful tools that offer only a command line interface. You might also find yourself working with some remote machine that does not have a GUI at all. Make sure that you can do the basic things, such traversing the file system, creating or removing files, viewing their contents and running CLI tools.

Shell scripting

Once you feel comfortable with using the command shell, you can save yourself a lot of time by automating routine tasks such as deployment, infrastructure provisioning or running development tasks. Depending on your operating system, you should learn how to write either for Linux and OSX using Bash or for Windows using batch files or PowerShell. Often other software comes with a set of such scripts to use it, so being able to read and understand the underlying commands will be helpful as well.

Linux

Linux is a family of free open-source operating systems built based on the Linux kernel. There are different Linux distributions available, such as Ubuntu, Debian, CentOS, Mint, openSUSE and etc. You can find Linux distributions running on all kinds of devices, starting from embedded electronics and appliances to smartphones and servers. Linux is different from Windows in a number of ways, so if you come from a Windows background, you’ll need to learn some fundamentals, like how the file system is structured, how access control works, how to install software and etc.

For a web developer, it’s important to be familiar with Linux since it’s extremely popular operating systems to run web servers. There are a lot of books and learning resources available for Linux, however, to really get to know it, you should start using it. You can install Linux as a primary or secondary OS and use it for everyday tasks. Once you start, you will figure out the details along the way.

Networking

Since we’re talking about web development, networks will play an important role in our lives. The users of your application will access it using the Internet over a cable or wireless networks. You application itself might communicate with other services over a network. To understand how this works, it’s important that you learn some of the basic networking technologies.

Protocols

A protocol is a set of rules on how applications should communicate with each other. They describe how a connection should be initialized, the order and the format of messages sent. There are several important protocols that are frequently used in web applications.

  • Hypertext Transfer Protocol (HTTP) is the underlying protocol of the World Wide Web and defines how browsers or other clients can request resources from web servers. When you enter a URL of a website in a browser, it actually makes one or several HTTP requests to the server to load the HTML, CSS, scripts and other assets. Its counterpart, Hypertext Transfer Protocol Secure (HTTPS) allows encrypting the transferred information to allow for secure communication.
  • HTTP/2 is a major update to HTTP that aims to improve the performance of web applications. It enables multiplexing, meaning you can transfer different content over a single open request at the same time. The protocol enforces encryption and minimises the protocol overhead by compressing header fields.
  • Transmission Control Protocol (TCP) is a transport layer protocol that allows to send packages between different applications in a reliable, ordered and error-free way. HTTPS and HTTPS work on top of TCP.
  • Internet Protocol (IP) is an internet layer protocol that defines how a package must be routed from one host to another. TCP uses IP for routing. There are two major versions of the protocol currently in use: IPv4 and IPv6.

Domain Name System

The Domain Name System (DNS) is responsible for translating human-readable names of Internet resources into their corresponding IP addresses. Resolving a domain into an address is quite a complicated process under the hood, and you will need at least some basic knowledge on how it works in order to be able to configure your application correctly.

Web Servers

Web server is a special type of software that is responsible for listening for incoming connections using different protocols and responding with the requested content. A web server can serve static files, such as HTML, JavaScript files or images, run dynamic scripts written in different languages or serve as a proxy for processes running as separate applications.

Web servers can also play an important part in improving the performance of your application by enabling compression, caching and load balancing.

These are some of the more popular web servers.

  • Apache HTTP Server is a free open-source web server developed under the Apache Software Foundation. It’s a fast and powerful web server that can be configured to serve static content, but is also frequently used to run scripts written in PHP, Python and etc.
  • Microsoft IIS is a general-purpose web server from Microsoft that is available on the Windows Server operating systems. IIS is a popular choice for companies using the Microsoft technology stack.
  • NGINX is web server that’s usually used as a high-performance static content server or a proxy for dynamic applications. In contrast to Apache, it cannot serve dynamic content on its own, but can sometimes achieve better performance. NGINX is a free and open-source web server but offers a more advanced paid version called NGINX Plus.

Network Troubleshooting

Sometimes you might find yourself in a situation that your application or web server is for some reason unavailable. Although debugging such cases might be tricky and there can be a lot of reasons for outages, there are some tools you can use to try to diagnose the situation and get to the source of the problem.

  • Ping is a cross-platform CLI utility that can be used to verify if a particular IP address is reachable. It also measures package loss and the roundtrip time that will give some idea about the health of the connection.
  • Ncat is a powerful Linux CLI tool, a Swiss army knife of networking tools, that allows you to send messages using different protocols, scan for open ports, create tunnels and much more.

Virtualisation

Virtualisation is an important concept in computing that has radically changed how resources are managed. Using virtualisation, we can create separate servers, workstations, files, networks or disks on a single piece of hardware. To the users, all of these resources appear as real physical resources, however, they are really provisioned using software, abstracted from the underlying hardware.

Virtual Machines

One of the most frequent use cases for virtualisation is the need to run different software or even different operating systems on a single physical machine. These are called virtual machines and provide completely isolated environments with its own memory, disk space and software installed. Virtual machines can be used for development purposes to run different operating systems on your workstation, but more importantly, to run multiple applications on a single hardware server without interfering with each other. Software for creating and running virtual machines is called a hypervisor.

  • VirtualBox is a free open-source cross-platform hypervisor currently being developed by Oracle. It supports creating and running virtual machines, as well as sharing resources, such as folders or clipboard, with the host machine.
  • Parallels Desktop for Mac allows you to seamlessly run Windows applications on your Mac. Parallels is convenient if you need to set up a Windows development environment on your Mac or to test something on Windows.

Containers

Containers is a different virtualisation mechanism to run software, but unlike virtual machines, containers don’t run a complete operating system, but share the same container runtime. This makes them much more lightweight and quicker to start, meaning that you can run many containers on a single machine without wasting a lot of resources. Best practices for containers is to run each piece of your application, such as a backend service, a web server or database, separately from the others, providing more granular control and isolation.

  • Docker is the most popular cross-platform container runtime developed by Docker, Inc. With Docker you can create a container image that is needed to run your application using a Docker file. These images can then be published into a registry to be run by others or deployed to a web server. Using Docker you can easily create a clean, pre-defined, disposable environment to develop your application and then use the same image to run it on production. There is a wide range of tools available to manage images and running containers, as well as services that allow you to host containerised applications.
  • Kubernetes is an open-source platform for deploying containers on a cluster of machines. It allows you to define services based on container images and configuration parameters, as well as create virtual storage or network resources. It also provides operational capabilities such as scaling, load balancing and logging. You can host Kubernetes on your own machines, or use one of the cloud-based services. It’s an extremely flexible and powerful platform that is currently rapidly evolving.
    In this chapter, we’ve focused mostly on tools and platforms that can be useful for developers on different platforms. However, when you start working with a particular platform, you will discover a lot of new tools that will be useful in your particular context.

Cloud Deployment

Once you're done developing your applications, it's time to deploy it. There are multiple cloud providers that offer all the necessary services to build and deploy your applications, the largest of them being Amazon Web Services, Microsoft Azure and Google Cloud Platform. There are also smaller players that provide more tailored services that can be easier to set up for beginners.

Why clouds? We believe, that deploying an application into a cloud-based service nowadays is the best way to go. Pricewise you might find better offers from local data centers available in your region, but joining a larger cloud provider will grant you access to a lot of modern services and different geographical regions. Unless you have compelling legal or financial reasons to host in a local datacenter, clouds seem like a better alternative.

When we're talking about application hosting, there are several layers of abstraction you can choose, from infrastructure as a service, meaning virtual machines, disks, and networks, to containers as a service, a high-level abstraction for running containers over a cluster of machines. Lower abstraction levels give you more control over the setup, but also require more work and technical skills to configure, while higher-level services make things easier, often reducing the level of control and transparency.

Map of cloud deployment platforms

Infrastructure as a Service

Infrastructure as a Service (IaaS) refers to services that allow you to run virtual machines, file systems, and virtual networks. To make onboarding easier, some platforms provide you with pre-configured images of machines built for different purposes, such as an Apache server on Ubuntu or a PostgreSQL server. You can create a VM based on one of those images and then tailor it to your needs, for example, install software, configure networking and setup security and firewalls. Although IaaS gives you the most control over your services, it's also requires the most efforts to set up. It's a good choice when you need to host something other than a plain web application.

Here are some of the popular IaaS services from different providers:

  • DigitalOcean is a cloud platform that offers IaaS services in data centers worldwide. Setting up a VM in DigitalOcean is relatively simple, however, it only offers Linux virtual machines. DigitalOcean offers $100 worth of credits for 60 days to try their services.
  • Amazon EC2 is the Amazon's IaaS offering and a leader in the overall market share. EC2 offers different kinds of virtual machines with flexible configuration, however, the initial setup can be more challenging. It offers a free 12 month tier for some of the smaller VM types.
  • Azure Virtual Machines is an IaaS offering from Microsoft with a large number of pre-configured images. Microsoft offers $200 worth of credits for one month and 750 hours of small VM's for 12 months.
  • Google Compute Engine is a service in the Google Cloud Platform that offers $300 credits for a 12 month trial period, an always-free micro VM instance.

Platform as a Service

Platform as a Service (PaaS) is a type of service for running web applications designed for easier setup. Unlike IaaS, PaaS services provide a complete technology stack to run your application without the need to set up the environment on your own. Some of the other concerns that are usually handled by PaaSes are:

  • A user interface for managing, running or stopping your application;
  • Basic monitoring and logging capabilities;
  • Horizontal and vertical scaling to cope with increasing loads;
  • Environment configuration;
  • Health checks to make sure your application is running and accessible.

Some PaaSes have a limited set of languages they natively support, but nowadays most of them support running Docker containers, meaning you can run an application written in almost any technology. PaaSes are usually designed for running web applications, and might not be suitable to run daemons or other kinds of software.

Some of the popular PaaS offerings are:

  • Heroku is one of the oldest PaaS services out there. It supports running applications written in Ruby, Java, PHP, Node.js, and others, as well as deploying applications in Docker containers. Heroku offers a free tier, however, your application will be put to sleep after 30 minutes of inactivity.
  • Netlify is a service for hosting static websites. It offers a fully fledged free tier with support for automated deployment, HTTPS, redirects, A/B testing, form submission and backend logic using AWS Lambda functions.
  • Azure App Service is an offering from Microsoft Azure. It allows hosting applications written in different languages on both Windows or Linux, as well as running Docker applications. There's a free shared instance available for the Windows app service.
  • Google App Engine is a service on Google Cloud that supports deploying applications written in Java, Python, Node.js, Go and PHP and Docker applications. There's also a limited free tier available.

Function as a Service

Function as a service (FaaS), sometimes also referred to as serverless computing, is a relatively new concept in cloud computing. It's designed to deploy simple atomic functions into a runtime without the need to worry about the infrastructure and a lot of operational concerns, such as logging and monitoring. Such functions can be invoked as regular services via HTTP requests, but a lot of platforms also provide the ability to react to various events such file uploads, a message in a queue or a modification in a database. The pricing model of FaaSes is also different. Instead of uptime, you pay for execution counts and consumed resources. This makes it a financially attractive solution for parts of your application that don't need to be up all the time or need to scale dynamically.

Functions have their downsides though. The tooling for most platforms is still in its infancy and local development may be not as smooth as with regular applications. The situation, however, can improve as the serverless movement is gaining momentum.

Popular cases for using functions include simple backend services that can be implemented with a minimal amount of code or post-processing to different events in your applications. Most large cloud providers also have a function offering:

  • AWS Lambda offers integrations with other Amazon services such as S3, DynamoDB, Kinesis, SNS, and CloudWatch as well as the ability to orchestrate function-based workflows using Step Functions. The AWS SAM tools allows testing your functions locally before deployment. The free tier offers 1M requests and 400,000 Gb seconds computing power per month.
  • Azure Functions is a similar offering from Microsoft that allows integrating with Azure services, such as Blob Storages, Storage Queues, CosmosDB and etc. Azure Functions can be integrated into Logic Apps to form complex workflows. It currently supports C#, F#, JavaScript, and Python as well as Java but in preview. It also offers 1M requests and 400,000 Gb seconds computing power per month.
  • Google Cloud Functions provides similar integrations to Google Cloud services, however, only JavaScript and Python functions are supported at the moment. Local development is supported only for JavaScript functions.

Containers as a Service

With the rise of containers and Kubernetes as the dominant orchestration platform new "Kubernetes as a service" solutions are rolled out by all the major cloud platforms. Such services are commonly referred to as Containers as a Service (CaaS). Since hosting your own Kubernetes cluster is not a trivial task, deploying to a managed cloud cluster is a better alternative for most cases. Such services offer maintained control planes, upgrades, guaranteed cluster uptime and middleware so you can focus on developing your application.

  • Amazon EKS provides a managed Kubernetes cluster with control nodes deployed to multiple availability zones to ensure uptime, built-in security features, and the ability to integrate with other AWS services. However, the process of setting up a new cluster is not as straightforward as it could be. There is no free tier available, and you need to pay both for the worker nodes, and a fixed fee per cluster.
  • Azure Kubernetes Services (AKS) offers a relatively easy initial setup process and a competitive pricing model, that doesn't charge anything for managing the cluster, but only for node VMs. 
  • Google Kubernetes Engine has received a lot of positive reviews from the community, which is to be expected from the company that created Kubernetes. It's praised for its simple setup and operations. It does, however, offer less integrated services than other platforms.

The cloud landscape is actively developing and providers are racing to offer new innovative services and competitive pricing. Keep an eye out for new offerings from major cloud platforms but also follow smaller platforms that offer more tailored solutions.

Conclusion and Next Steps

Now that you have an overview of what web development is about, it's time to take the next step. As the English biologist Thomas Huxley stated:

Try to learn something about everything and everything about something.

For a developer, it's a great benefit to have some understanding of different technologies, but before you reach this level, you should pick a stack and devote yourself to it completely. How do you do that? There's no correct answer to that question and different approaches are possible. Some people take up frontend development because they enjoy creating visual representations. Some people have a project in mind and pick the easiest way to implement it. Others take up courses that lead them to their career path.

Whatever path you take, immerse yourself in it completely and make sure you enjoy this path. With enough persistence and motivation, you will learn to be a fully fledged web developer in no time!