Software development is the process of creating a piece of software designed to perform a particular task. It encompasses the entire software development life cycle. This resource is designed to give you an overview understanding of what software development is and the most important components, terms and trends in software development today.
If you want to delve deeper into any of the software development sub-topics covered here you’ll find links to dedicated blogs so you can explore further. We hope you find this a great starting point to learning more about software development and we’d love to hear from you if you think we can improve it further!
Software is a computer programme – a structured compilation of commands computers and other hardware understand and act on because they are written in computer programming languages consisting of binary code. Software is developed to complete a specific task like automating a previously manual process to make it more efficient, organise and process information (data) in a certain way or providing entertainment, like a video game.
Most software has a front end and a back end. The front end is the part users see and interact with. It is considered the ‘presentation layer’. The back end is the plumbing behind the scenes that supports the front end and allows it to work the way it does. It is the ‘data access layer’ or ‘infrastructure layer’.
Front end and back end development involves different skill sets and technologies and software developers are usually classified as front end, back end or full stack developers. Full stack developers have the skills and experience to work on both the front and back ends of software being developed.
K&C - Creating Beautiful Technology Solutions For 20+ Years . Can We Be Your Competitive Edge?
Drop us a line to discuss your needs or next project
The terms software development and software engineering are often used interchangeably. But is there a difference? Some might argue differently but, technically, there isn’t any officially defined difference.
The Merriam Webster dictionary defines software engineering as:
“A branch of computer science that deals with the design, implementation, and maintenance of complex computer programs.”
But the dictionary does not define software development, suggesting software engineering instead.
The Cambridge dictionary defines both software development and software engineering in the same way:
“the activity of creating computer programs”
Computer programming and coding are other terms sometimes practically used as similes for software development. The distinction that could be made is software development refers specifically to the end-to-end process of creating a software product. Computer programming and coding is part of that process. A good analogy would be the distinction between ‘writing’ and ‘writing a book’.
There are many different types of software and the broad category can be broken down into different groupings, like use cases or technical qualities. At a macro level we can break software down into four main groups:
Application software is the most familiar category and refers to software that runs on a computer or mobile device. A computer game, Microsoft Office, accountancy software, Slack, Skype, Google Maps, or a web browser like Chrome are all examples of application software.
Application software can be native – designed to run on either a particular operating system like Windows or macOS (desktop and laptop) and Android or iOS (mobile devices).
Or it can be cross-platform. Cross-platform application software runs on web browsers. Some cross-platform applications, usually mobile apps, are referred to as ‘hybrid’ because they run on a browser but deployed in a native container. The approach gives cross-platform apps a ‘native’ feel and can allow for better integration with the device’s hardware.
Application software is usually designed to help its users complete a particular task, including entertainment. The four main sub-categories of application software are:
Custom software is developed as a tailor-made solution to the needs of a particular user. For example, a solar plant operator might want to develop software that will track and record yields across its installations. It has to be custom developed to the unique requirements of the specific conditions and commercial priorities.
Custom software is usually developed to provide a competitive advantage in a way a one-size-fits-all commercial software can’t.
Commercial software is developed as a solution to a particular market or consumer need and designed to be resold to a theoretically unlimited number of users with the same need. Video games are a good example of commercial software, as are mobile apps and SaaS products like accountancy or bookings software designed for restaurants, hotels or sports facilities.
Sometimes commercial software is also designed to customisable to specific customer needs. Hybrid customisable commercial software usually offers a standard version of the product designed to fit the most general needs. Customers who want to customise that to their requirements then pay for the additional development work in addition to the usual license fee. Salesforce is one of the best-known examples of commercial software that can be customised.
Open source software is developed and released by the copyright holder under a license that gives users the right to use, analyse, customise, develop on, and distribute it to anyone for any purpose. The source code of open source software is often developed collaboratively by a public community of developers.
Improvements suggested by the open source community are integrated into the software by consensus decision. WordPress, the Mozilla Firefox browser and VLC Media are all popular examples of open source application software. Other kinds of software, like operating systems and programming languages, frameworks and software development tools can also be open source.
The term open source was introduced by The Open Source Initiative (OSI) in the late 1990s.
‘Free source’ is a term sometimes used for open source software distributed for free without a license fee.
Cloud native software development means building software in a way that is specifically customised to running on a cloud computing infrastructure, including public, private and hybrid clouds.
Cloud native software can either be developed as ‘cloud agnostic’, which means it can run on any kind of cloud, public or private, with minimum reconfiguration or as native to a particular cloud computing platform like AWS, Microsoft Azure or Google Cloud Platform (GCP).
System software is an umbrella term for software designed to start and run computer hardware and allow application software to run on it. It acts as a facilitator to the optimal interaction between hardware and application software running on it.
There are several different categories of system software:
A device’s operating system is probably the kind of system software we are most familiar with. It is a kind of software application that sits between a computer’s hardware and the application software that is installed on it. Applications then make use of the hardware by sending requests for particular services, like processing or visual representation, to through the OS, communicating with it by API.
Users of the device an OS is installed use it to communicate with the hardware without having to know a programming language. For example, when you turn off your laptop’s power by selecting ‘shut down’ via Windows or macOS.
The operating systems we are most familiar with (Windows, macOS, Android and iOS) as users of mainstream tech like laptops, smartphones and tablets use a graphical UI (GUI) user interface. Linux, popular with software developers, has a command-line interface (CLI), which means you don’t tap a button with a cursor but type in a command.
The most common operating systems in 2021 (used by almost 92% of all devices) are:
Utility software is also designed to perform particular tasks but in the background and related to hardware performance and security. Examples of utility software are antivirus programmes, hard disk optimisation tools and device backup.
Device drivers are a kind of software extension, or component, that facilitates communication between an operating system and device. If an application needs to access data from the device it is running on the driver communicates that request to the hardware, which then returns the requested data to the operating system which, in turn, passes it over to the application.
Middleware is a category of software that augments the services and capabilities the operating system provides and is often used to handle things like API management, data management, authentication, messaging and more advanced application services.
RedHat defines the role of middleware succinctly as:
“Middleware helps developers build applications more efficiently. It acts like the connective tissue between applications, data, and users”.
Middleware has become increasingly important with the rise of the DevOps software development methodology and cloud-native development because it helps manage the complexity of applications deployed across hybrid and multi-cloud infrastructures.
As part of a DevOps approach to application software development, middleware is used to ensure software developed in one environment works in the same way when run in another.
Embedded software (also referred to as firmware) is a kind of software designed to run on hardware that wouldn’t normally be defined as a computer. For example, the software in a car or a robot vacuum cleaner are examples of embedded software. As is software developed for robotics like those used to automate production lines or IoT (internet of things) devices like smart doorbells or virtual assistants (Alexa, Apple HomePod, Google Assistant etc.).
Embedded software engineers need specific skills and knowledge application software specialists often don’t, including be able to read schematics and component data sheets, communication protocols between processors and other components, ability to convert between decimal, hexadecimal and binary and use bit manipulation.
Developing embedded software involves the use of a cross compiler that converts code developed on a computer into a format executable on the target device. Debugging is also different and requires the use of an in-circuit emulator and specialist hardware.
Artificial intelligence algorithms are a kind of software that is usually a component of a larger piece of software that could be application software, system software or even embedded software.
AI software is an algorithm, or collection of algorithms, that are sets of rules, or step-by-step procedures for calculations. The algorithm applies the rules it has been given to a data set, which allows it to learn on its own.
For example, a simple algorithm to find the largest among three different numbers would look like:
Step 1: Start Step 2: Declare variables a,b and c. Step 3: Read variables a,b and c. Step 4: If a > b If a > c Display a is the largest number. Else Display c is the largest number. Else If b > c Display b is the largest number. Else Display c is the greatest number. Step 5: Stop
Machine Learning and Deep Learning are the two types of AI most used in software development.
Machine learning algorithms parse data, isolate patterns that exist, ‘learn’ from these and decide on or make a prediction about something in reality connected to the database learned from or similar data. ML algos are based on decision tree learning, inductive logic programming, clustering, reinforcement learning, Bayesian networks and other processes that resemble some of the ways we learn as humans.
Deep learning is an approach to AI that is more like how human intelligence works. Neural networks break the decision making process down into chunks and pass it through a system of checks and balances. If we see a Stop sign partially obscured by a branch, we may not see the whole sign but will be confident it is a stop sign because we can see what shape, size and colour it is and part of the word written.
A deep learning algorithm assigns weightings to the information it does have in the context of the database it has learned from and reaches a ‘probability vector’, or, in other words, an educated guess.
Software is created through a development life cycle that typically includes phases of ideation, concept validation, specifying requirements, design, prototyping, coding (programming), documentation, testing, bug fixing, increasingly operations – deployment into the live environment and making sure the software runs as expected (DevOps), maintenance and the release of new iterations. The number and how the different phases are named varies between development methodologies and frameworks.
While every software development process goes through a similar process or life cycle, there are important variations in the methodologies (sometimes also referred to as software development life cycle models) used throughout the process.
Software development methodologies can be switched between depending on the qualities and priorities of a project and have their perceived strengths and weaknesses in different scenarios. Agile software development, a flexible, iterative approach to the software development lifecycle, is the dominant methodology in 2021 but far from the only approach taken by software development teams and their managers.
Some of the main methodologies and methodology frameworks for software development are:
For more information on each of these currently and historically popular methodologies and frameworks, check out our dedicated post – Popular software development methodologies and frameworks .
As already covered, software development refers to the whole lifecycle that culminates in a live piece of software doing its job for users and not just the programming or coding phase. But writing the structured computer code that makes up a piece of software is at the core of software development.
Codeacademy explains programming with a useful metaphor:
“Put simply, programming is giving a set of instructions to a computer to execute. If you’ve ever cooked using a recipe before, you can think of yourself as the computer and the recipe’s author as a programmer. The recipe author provides you with a set of instructions which you read and then follow. The more complex the instructions, the more complex the result!”
Programming, or code, is written in a programming language computers that think in binary can understand. Humans, however, can’t typically understand these strings of 1s and 0s. Programming languages act like a translator which converts binary strings of 1s and 0s into a format humans can understand and write.
When a software developer learns a programming language they can write the structured sets of instructions software consists of. The computer then converts the programming language into the strings of binary it understands and executes on the instructions given.
Programming languages are classified as being either low level or high level. Low level languages are closer to binary. That makes them harder for humans to read and write but easier for computers. A software engineer coding in a low level language can give a computer more precise instructions understood and actioned on faster than if they were written in a high level language more abstracted from binary.
High level languages use more words we also use in human-to-human communication, like object, class, request, run etc., which makes it easier for humans to learn them.
As computers have become more powerful, the difference in the time needed to convert low and high level languages into binary has been reduced to milliseconds. That means high level languages can be used in most modern software products without any meaningful impact on performance.
Programming paradigms classify programming languages based on their features. The main paradigms are:
Groups coded instructions by the part of the state they operate on.
The intended result is declared as the value of a series of function applications.
The coder declares the properties required of the desired result but not how it is computed.
The coder tells the machine how to alter its state.
Coded instructions are grouped by the procedure they form part of.
Written in a way that is very similar to formal logic, expressing facts and rules. Rules are written as clauses and read declaratively as logical implications eg.
H if B1 and … and Bn.
The desired result is the optimal element from a range of choices based on given criteria. For example, if the optimisation problem was arriving at the number 500 by using the numbers 5, 100, 10, 9, 8, 2 and 2, the desired result would be 5×100 and not 9×2-8/2*100.
The desired result is declared with data streams and change propagation.
The most commonly used contemporary programming languages usually don’t fall neatly into a single paradigm but combine features from several.
The terms framework and library are sometimes used interchangeably but there is a difference with libraries offering greater flexibility. A good analogy for the difference between a library and framework is that the former is like designing an IKEA kitchen. You can create a kitchen of almost any shape or size with a lot of choice between elements and how they can be combined.
A framework is more like building a modular house. You have choices in the architectural layout and design but they are limited. The kitchen and bathrooms, for example, should be positioned close to where the main water pipes run top to bottom and the kitchen should be on the ground floor. You have a limited amount of space to divide up into rooms etc.
More technically defined, the difference between a library and a framework is something called inversion of control. When a software developer uses a library they control the flow of the application – when and where the library is called.
When a framework is used, it is in charge of the flow and tells the developer where they can plug in their code to customise it. And it calls the code plugged in as it needs.
There are a large number of frameworks and libraries, especially for the most popular programming languages. However, a much smaller number are regularly used commercially. As of 2021, some of the most popular frameworks and libraries in use are:
Node.js is a popular back-end run-time environment also sometimes referred to as a framework though it technically isn’t.
The most used python frameworks are Django, Flask and Bottle, while the most used libraries are TensorFlow, SciKit Learn, Numpty, Keras and Pytorch.
The most used Java frameworks are Spring, Scala, GWT (Google Web ToolKit), JSF and Struts.
The most used Java libraries are Lombok, Guava, Hibernate, Feign and Spring Boot.
K&C - Creating Beautiful Technology Solutions For 20+ Years . Can We Be Your Competitive Edge?
Drop us a line to discuss your needs or next project
As well as frameworks and libraries, software developers make use of a large collection of software development tools designed to make their job easier. Tools are used throughout the software development process to help develop, debug, maintain, or otherwise support software.
Popular examples of software development tools are Integrated Development Environments (IDEs), which are really a collection of tools, Git for version control, Selenium and Cucumber for integrated testing, Stack Overflow an online community of developers where knowledge and resources are shared, the continuous integration server Jenkins, and many more.
Even Docker, the containerisation platform, and Kubernetes, the container orchestration platform, used in cloud native development, could be described as software development tools.
A software development kit (SDK) is a toolkit that comes in a single installable package and typically include a compiler, debugger, often a framework, and other tools. SDKs are usually made to help software development for a particular combination of hardware platform and OS.
For example, the iOS SDK is designed to help developers creating iPhone and iPad applications and AWS provides SDKs that provide language-specific APIs for AWS services.
A small app can be built in its entirety by a single full stack developer with both backend and frontend experience. But most commercial software products are built by a software development team consisting of several different roles.
Some roles, like Scrum Master, are specific to a particular software development methodology (Agile), while others are more general. Not every development team will consist of the same collection of roles and the number of specialists performing each role will vary with a project’s nature and scale.
Some of the most common roles found in a modern software development team are:
Oversees a software development project from a bird’s eye view with a strategic focus on budget, risk, schedule and resource management. Project managers often don’t have a software engineering background themselves.
Product owners also an analytical not technical role and is focused on a product’s functionalities and features. Their main responsibility is that a software product is fit for purpose and its business value optimised. In an Agile development model, the product owner also prioritises the order in which new features and functionalities are developed and released after the launch of the MVP.
The tech lead of a software development project is responsible for selecting the best technical solutions to meet business requirements. This role used to often be referred to as the software architect.
In consultation with other stakeholders, the tech lead makes decisions on things like what programming languages or frameworks the software is to be written in, infrastructure choices, perhaps what methodology the team will follow etc.
Full stack developers are generalist software engineers that have a working knowledge of a wide range of frontend and backend skills and technologies. A full stack developer can theoretically build a whole software solution alone. But full stack developers usually aren’t specialists so are often seen as most valuable in teams working on less complex software.
Front end developers are specialised in and work exclusively on the front end of a piece of software being created by a development team. They use front end languages, libraries and frameworks to create the visual interface users see and interact with.
Back end developers are specialised in and work exclusively on the back end of a piece of software being built by a software development team. The back end team are responsible for a software’s CRM, databases and server-side web application logic and integration.
The quality assurance (QA) lead is accountable for the overall quality of deployed software. They manage the QA team itself and makes sure its findings are acted upon by those responsible for different areas of the software development process.
The QA team is made up of QA engineers who run software testing throughout the development process, identifying potential issues and reporting them back to the responsible teams. Most testing and QA assurance is now automated as part of a DevOps approach, especially for more complex software products and consists of regression error testing ie. have new features resulted in errors or bugs in already existing parts of the software?
Testers are usually responsible for manual verification software works as it is supposed to for end users. Standard user journeys can usually be tested by automated tools but manual testers take an analogical approach to make sure bugs and errors don’t crop up in alternative user paths and exceptions. For example, in airline ticket booking software, can the system handle the exception of a customer booking outgoing and incoming flights on the same day?
A UI (user interface) designer is responsible for the user-facing appearance of software. The UI designer designs all the screens a user might see while interacting with software, including all the visual elements, and their interactive qualities, that take a user from one screen to another.
A user experience (UX) designer is responsible for software being as intuitive as possible for users. Nobody wants a kitchen that looks beautiful but in which elements are inconveniently positioned. The sink should be adjacent to a work surface and not too far from the hobs etc.
A UX designer makes sure software doesn’t just look nice is convenient to use.
When does IT Outsourcing work?
(And when doesn’t it?)