- emailSend a requestphoneCall +1 (650) 662-5052phoneUK +44 2035140925phoneToll-free 855-258-67-67
- Services
- Expertise
- Portfolio
- Testimonials
- About Us
- Contact Us
Data Engineering Services







Leverage Our Expertise and Boost Your Project
Services

DATA LAKE AND DATA WAREHOUSE
Get a unified view of operational data across your business. Altoros will help you to drive business intelligence by quickly aggregating, integrating, structuring, and storing data from disparate sources. With current and historical data gathered in a single point, you get a consolidated view of your business processes.
DATA MIGRATION
With NoSQL and NewSQL, we migrate and store your large data volumes to help you achieve high availability and scalability of your system. Our services include migration to modern cloud architectures and data storages. We launch new suitable infrastructures, prepare data for migration, optimize it, and migrate it securely.


CONTINUOUS INTEGRATION AND DEPLOYMENT
Enable smooth integration of custom applications with third-party systems and aggregate data from multiple sources for further analysis. At Altoros, we integrate disparate data sources by using the extract, transform, load (ETL) best practices. Through our data integration services, our engineers and consultants will help you to maintain consistency of your data landscape, as well as guarantee an error-free data flow.
DATA ANALYTICS AND VISUALIZATION
Build custom data management solutions to visualize and analyze data. Our experts can help with building an ETL/ELT pipeline to efficiently process data. With Altoros, you can increase the visibility of your business processes and make smarter decisions. We will help you to set up and build web-based reporting solutions that provide a real-time view of your data, while simplifying the process of making informed decisions.


DATA STORAGE AND ETL PROCESSING
Perform data cleansing, profiling, normalization, extraction, transformation, as well as enable data processing, mining, upload to data warehouses, and more. We will help you to design ETL tools based on Hadoop HDFS, Amazon S3, GCP Cloud Storage, Azure Blob Sore, and the DSE distributed file system.
DATA PIPELINE DEVELOPMENT AND IMPLEMENTATION
Building a scalable data pipeline is a complex without the right combination of skills and experience. Leverage our team's experience architecting and implementing data pipelines for businesses of all sizes across industries. We will make sure your data flows are designed accurately and reliably, so you can stay focused on driving your business-critical tasks.

Let's discuss your project
Contact Altoros’s expertsDevelopment
Data engineering
Altoros has first-hand experience in NoSQL databases: our experts assisted the core teams in the development of data stores, integration with other technologies, and fine-tuning performance.

Our engineers developed the NuoDB Migrator, which enables users of traditional SQL databases to easily migrate their data to NuoDB.

The Altoros team contributed to the core of the product.

Altoros benchmarked a number of Redis-based products: Redis Cloud, ElastiCache, openredis, RedisGreen, and Redis To Go.
Big data expertise at Altoros has been acknowledged by Clutch (ex-SourcingLine), a research company based in Washington, DC, for two years in a row:




WHY PARTNER WITH US
Find out more about the technology
Contact Altoros’s expertsRelated cases
The team at Altoros has successfully implemented 1420 projects, some of which can be accessed through this page. We do also share more details on a particular project and other stories of success on demand. Please do not hesitate to reach out to us with a request!


A New-Generation Energy Management System
Description
Galtronics wanted to support the Green IT technologies by providing their users with a tool that could help to analyze power consumption without digging through paper bills. The company turned to Altoros to build an energy management system for collecting, storing, and processing petabytes of power consumption data. The solution also had to easily scale up from a single server to thousands of machines without compromising the overall performance.
The outcome
Altoros suggested using the Cassandra NoSQL database and the Hadoop distributed computing system for building a highly scalable system storing and processing vast amounts of data. Cassandra can write to a data store 2,500 times faster that MySQL solutions. Cassandra enables power consumption data to be gathered and aggregated in a fraction of a second. Hadoop was implemented to cope with the big data management challenge. It uses clusters of computers and distributed processing to analyze petabytes of power consumption records. The solution makes it possible to collect, aggregate, and process power consumption data in real time. symHome also generates detailed reports on how electricity is being consumed. The Web site can be accessed from any PC or Apple device. Users can quickly identify which appliances and devices cost them most money. The system can be scaled next to endlessly as the number of connected houses grows. The solution helps the customer achieve its Green IT goals by continuously reducing the environmental impact of their products and operations.
Technology stack
Client Platform/Application Server - Apache Tomcat 6.0
Technologies - Spring Framework 3.0, Quartz, JSP 2.1, Adobe Flex SDK 3.5, Apple iOS SDK 4.3, Hadoop
Programming Languages - Java, ActionScript 3.0, Objective-C
Database - MySQL, Cassandra
Development Environment - Eclipse, Flash Builder 4, Xcode 3.2.6




The Cloud-based Document Exchange System
Description
The customer is a start-up company that came to Altoros with an idea to develop a service similar to “virtual FedEx” that delivers electronic documents. The main advantage of the emerging system was to be a simple and intuitive user interface. Ease of navigation and availability of the planned features were tested on a prototype developed by Altoros.
The outcome
It was decided to use cloud technologies to ensure high availability, quick response times, and easy horizontal scalability. The system was integrated with EMC, FileTransfer services, and a third-party Reporting Tool selected by Altoros’s experts.
The customer released a new solution that makes document exchange easier and more efficient. The system has already become in demand, thanks to a user-friendly interface, smart navigation, enhanced security, and unlimited storage space in the cloud. By testing prototypes, we were able to carefully study all requirements, confirm functionality, and access usability before we started development.
Technology stack
Server Platform - CentOS
Client Platform/Application Server - CentOS
Technologies - Couchbase, Solr, Pentaho, Spring MVC, jQuery
Programming Language - Java
Database - Cassandra
Database Design Tool - yEd
Development Environment - Eclipse




The NuoDB Migrator for Moving SQL Data to a NoSQL Database
Description
NuoDB is a NewSQL database platform with a unique geo-distributed database management system. It provides linear scalability, outstanding scale-out performance, continuous availability and can run anywhere—in a local data center, in multiple data centers around the world, or in the cloud. With support for multiple platforms, users are able to build hybrid solutions that have both Linux and Windows hosts.
The outcome
The NuoDB Migrator enables users to extract data from SQL-compliant databases and load it to NuoDB. The tool is able to manage large data processing scenarios, such as failures, continuation, restart and catch-up. During operation, completion status is displayed for task execution and users are provided detailed reports on the migration process.
While many enterprises would prefer to move to scalable NewSQL databases like NuoDB, the difficulty of data migration has held back progress. But with NuoDB Migrator and NuoDB Applier for Tungsten Replicator—developed by Altoros—it is now easy to jump on board with NuoDB’s geo-distributed database. The NuoDB Migrator is cross-platform and ensures a high level of data consistency and integrity, eliminating the old complications of data migration.
Technology stack
Server Platform - Unix, Linux, Windows, MacOS
Technologies - JDBC, Maven, Git, Travis, BSON, CSV, XML
Programming Language - Java 6
Database - NuoDB, MySQL, PostgreSQL, MSSQL, Oracle, NoSQL
Development Environment - IntelliJ IDEA





Independent Infrastructure Performance Benchmarking
Description
The cloud infrastructure provider turned to Altoros to do independent performance tests on their virtual machines and provide recommendations on how to make the system more efficient. The results of our assessment revealed that the system’s performance was in fact 20-30% higher than the results provided by the customer. Our engineers also drew up a list of recommendations on how to improve the system’s efficiency and gain competitive advantage.
The outcome
According to Altoros’s tests, a virtual machine with Ubuntu Linux installed processed 1 TB of TeraSort test data in 13.65 minutes, which is 1.2 times faster than in the customer’s test results. Featuring enhanced CPU bursting and improved disk input/output speed, virtual machines with custom OS installed were able to complete the same task in 6 minutes, which is 2.75 times faster than the results demonstrated during the initial benchmarking. The tests revealed that non-optimized Linux machines become unstable, if a cluster exceeds a certain size. The reports, instructions, and scripts provided by Altoros can be later used by the customer’s team to replicate the test results or to improve the system’s stability.
Technology stack
Server Platform - Ubuntu, CentOS
Client Platform/Application Server - Ubuntu, MacOS, Windows
Technologies - Apache Hadoop, Ganglia, Opscode Chef
Programming Language - Ruby, Bash, Scala
Development Environment - Ubuntu, MacOS, Windows




LikeFolio: Best practices of Cloud and Ruby development for Application Optimization
Description
LikeFolio shows users information on the top publicly traded companies most talked about on their networks. It also predicts how a portfolio of their securities worth $10,000 would perform over a period of 12 months. SwanPowers, LLC turned to Altoros to apply best practices in cloud and Ruby development to their existing system in order to satisfy the high requirements to the level of concurrency, load, response times, etc.
The outcome
Having analyzed the infrastructure, software architecture, and application code, we were able to implement improvements that have resolved all the issues mentioned above. The optimization included reconfiguring the Amazon EC2 instances, implementing a Redis pub/sub mechanism to decouple jobs from DB operations, managing social networks quotas internally, so that the application would never exceed the allowed number of requests, and much more. Thanks to the improvements introduced by our team, the customer was able to launch several marketing campaigns without being afraid of performance issues due to increased traffic. The application’s overall uptime and performance have been improved significantly. Taking into account the LikeFolio’s focus on social networks, Altoros made sure that it complies with the required API quota/limits.
Technology stack
Server Platform - Amazon Web services (EC2, Route 53, CloudWatch, RDS), Redis
Technologies - Ruby on Rails, Sidekiq, Capistrano
Programming Language - Ruby
Database - Amazon RDS for MySQL
Client Platform/Application Server - Unicorn





The Solr-Based Search Engine Solution
Description
The system of a European Wi-Fi Internet provider was significantly updated. The team added a Google-like search, refactored the architecture, and updated queries to the database to improve performance. In addition, the new solution serves as an intelligent tool that analyzes user preferences and helps to improve promotional strategies and create targeted advertising and content. It can be used to evaluate quality of services, gather statistics, and build detailed reports.
The outcome
The system performance increased by 1,000 times due to updated architecture, optimized queries, and improved search. The customer was impressed by the suggested solution and the improvements they were able to achieve. In order to test system performance and feasibility of the project, Altoros decided to develop a prototype. The team used three servers provided by the customer in a master-slave arrangement with the Puppet management solution for configuration and management. We uploaded all the content to a different storage to separate user authorization functionality from search queries. Solr was used for advanced search through the content. As the result, users can now search content by text as well as by tags. The system features auto suggestion and various data filters. New information is regularly uploaded by the crawler that visits a list of specified Web sites. Then it indexes and adds all this data to the content database. Users can search content in three languages, including English, French, and Dutch.
Technology stack
Server Platform - CentOS 5
Client Platform/Application Server - GlassFish
Technologies - Solr, Apache Hadoop, Apache ZooKeeper
Programming Language - Java
Database - PostgreSQL
Database Design Tool - Sparx Enterprise Architect
Development Environment - IntelliJ IDEA, Apache Maven, Navicat




Web-Based System for Retailers
Description
A Web-based system that enables retailers to gather statistics on consumer demand for their products and adjust their sales strategy according to these figures. TapMap is a start-up company that needed a solution to synchronize retailers’ Point of Sale systems with online accounts. The solution had to include a Web system providing information on stock and prices. The information was to be updated on a daily basis and accessible to customers through free iPhone and Android apps. The customers would scan barcodes with their smartphone cameras to compare prices while the Web system gathers statistics and generates reports for retailers. Apart from that, searches and scans were to be displayed on a visual map in real time.
The outcome
Our team of Ruby on Rails developers analyzed the architecture of the solution and found a way to improve it. The connections between the objects were optimized to enhance the structure of the database. This greatly improved the overall performance and provided nearly endless scalability.
The customer was one of the three winners at the International EXPO 2011. The iEXPO featured a number of promising startups who presented their projects to the audience.
Technology stack
Server Platform - CentOS
Client Platform/Application Server - Web browser/Nginx + Passenger, Apache + Passenger
Technologies - Ruby on Rails 2.3.8, Sphinx, MongoDB, MapReduce
Programming Language - Ruby 1.8.7
Database - MySQL 5.1 (primary), MongoDB 1.8 (for statistics)
Development Environment - TextMate




Highly Scalable System for DNA Analysis
Description
The customer turned to Altoros to improve its biotechnology system that analyzes DNA samples for mutations on early stages. The legacy tool was able to de-duplicate only 1,000 samples maximum—due to memory and CPU limitations—and it still took hours (or even days) to process the pipeline. The goal was to fix performance bottlenecks as well as enable linear scalability for processing 10,000+ bio samples at a time.
The outcome
Altoros has delivered a highly scalable analytical system for de-duplication of genome samples—as a part of the customer’s analytical platform. Thousands of hospitals and laboratories worldwide use the system to detect DNA mutations, saving thousands of lives. The analysis takes minutes now, not hours; it allows for processing 10x more genome samples—compared to performance of the legacy system. Altoros’s engineers have also proposed a reference architecture for updating a reporting solution. Inspired by our recommendations, the customer went on improving the system with open-source data analytics technologies, which will eventually allow for saving thousands of dollars.
Technology stack
Server Platform - Linux
Technologies - Apache Hadoop (Cloudera CDH 5.2.1), MapReduce, Apache Spark (Spark SQL), bash
Programming Language - Java, Perl
Database - HDFS


Why Our customers trust us
Our clients speak
Our aim is to reach customer satisfaction. Explore some of our clients’ testimonials to learn the results of our productive collaboration.

About Altoros

Partners







About Altoros
Altoros is a professional software development company with headquarters in Pleasanton (USA), branch offices in Norway and Finland, and development centers in the USA, Argentina, and Eastern Europe. With 18-year IT experience and a strong team of full-stack software engineers and consultants, we help our clients to achieve unsurpassed quality at all stages of the web, mobile, and desktop application development. By providing multiple time-zone teams and different formats of working (onsite, remote, hybrid, etc.), we help organizations across the globe to gain sustainable competitive advantage through the adoption of innovative technologies.Research & Development
The R&D department within Altoros keeps track of the latest technologies available on the market from 2011. The main goal of the department is to learn how projects can be developed faster, better, more effectively, and more efficiently. Our studies are mostly focused on big data solutions, data science, cloud computing, and cross-platform development.Contact us now
To ensure your project is delivered on time

Copied
Copy to clipboard
-
Headquarters
4900 Hopyard Rd., Suite 100Pleasanton, CA 94588 -
Altoros Finland OY
Kyllikinportti 2,00240 Helsinki, Finland -
Altoros Norge AS
Tordenskiolds gate 2, 0160 Oslo, Norway -
Development center, Eastern Europe
9 Dombrovskaya St., Fl 5Minsk, 220140 -
Development center, Argentina
Buenos AiresAv. Federico Lacroze 2827,C1426CPP CABA, ArgentinaSanta Fe25 de Mayo 2884, S3000FUASanta Fe, Argentina