Trusted by Market Leaders in Education, Travel, Finance and E-commerce since 2007
We put excellence, value and quality above all - and it shows
A Technology Partnership That Goes Beyond Code
“Arbisoft has been my most trusted technology partner for now over 15 years. Arbisoft has very unique methods of recruiting and training, and the results demonstrate that. They have great teams, great positive attitudes and great communication.”
How Arbisoft helped AEU reduce crawltime from 168 hours to less than 4 hours
Arbisoft helped Advanced Energy United (AEU) reduce its data crawl time from over a week to less than 4 hours, using a custom-designed distributed crawling mechanism and intelligent data extractors capable of semantic analysis. This resulted in a 97.6% decrease in minimum runtime duration and significant cost savings. Arbisoft enabled AEU to provide updated information to their subscribers more quickly and make better data-driven decisions.
IndustryEnergy
HeadquartersBoston
PlatformWeb
“We now have more data than the largest online encyclopedia as a result of leveraging Arbisoft’s expertise. They’re definitely web crawling experts.”Eric FitzVice President, Engineering and Product Development
AEU
Advanced Energy United (AEU) works toward a clean, secure, affordable energy economy through advocacy, analysis, and education — powered by data. Their product, InsightEngine, aggregates public Utility Commission data nationwide, helping subscribers track trends and decide better. That's where Arbisoft comes in.
aee - How we fit together
To be competitive, InsightEngine required daily data crawls of 50 large PUC websites. This entailed massive overhead costs in time, performance, and API service expenses. There had to be a more efficient way of doing this–and they came to Arbisoft to find it.
We implemented a custom-designed distributed crawling mechanism that used automated intelligent data extractors capable of semantic analysis. We optimized their system and added the ability to clean up data in realtime, further cutting down time between data collection and data usability. We also provided a REST API that fed data to their user-facing Ruby on Rails app. The high performance API was initially implemented using Google App Engine and later transitioned to the Django REST Framework app.
