We recently had a migration from ACT to Salesforce. One of the migration steps was to create Account records in a staging database from ACT Contacts. The source had 24,000 records and there were matching rules on three fields: Address1, City, and Company Name.
It took 15 minutes to get through about 20,000 rows, but then, it started to run very slowly and ended up taking nearly three hours to complete. The staging database was just a basic database created in Microsoft Access.
We discussed how to make the job run a little faster and decided we would add indexing to the fields that were being used in our matching rules. We expected that this would improve the performance of the job -- and it did.
After adding indexing to the three fields, the job took less than two minutes to run. That's the power of indexing.