Jump to content

Whmcs Migrator (Beta) - Updated 2013-11-12


Cody

Recommended Posts

Just attempted an import, and no package data came through.  The packages themselves were created, but were not assigned to customers.  What info do you need?

 

EDIT:  Some came through, some did not.  I'm researching to see what the difference is.

 

EDIT 2:  The only customer to get a package associated correctly is my test account, user #1 in WHMCS.  And then only one package (out of maybe a dozen) came through.  I do see many packages created successfully in Blesta, but any package in WHMCS that did not have a non-zero quantity is showing in Blesta with 0 quantity.  If the package quantity in WHMCS was either "unlimited" or "0", it's now "0" in Blesta.

Link to comment
Share on other sites

I just ran the importer on my rather large database.  It worked this time from localhost instead of over the network.

 

Items in red did not import, items in green were confirmed imported, items in orange cannot be confirmed because the do not apply.

  1. Staff (including login passwords)
  2. Client Groups
  3. Clients (excluding login passwords)
  4. Credit Card Payment Accounts
  5. Custom Client Fields and Values
  6. Client Notes
  7. Contacts - Imported as clients, this can be an issue...
  8. Tax Rules
  9. Currencies
  10. Invoices
  11. Recurring Invoices
  12. Transactions
  13. Client Credits
  14. Support Departments
  15. Support Tickets (excluding attachments)
  16. Support Predefined Response Categories
  17. Support Predefined Responses
  18. Sent Emails
  19. The following Settings:
    1. Mail Delivery Type
    2. SMTP Host, User, Password, Port, and Security Type
    3. Invoice Days Before Renewal
    4. Invoice Reminder Settings
    5. Auto Debit Days Before Due
    6. Suspend Services After Due
  20. Calendar Events
  21. Products (become packages)
  22. Registrars (become modules/module rows)
  23. Servers (become modules/module rows)
  24. Services
  25. Domains (become services)
Link to comment
Share on other sites

The latest version of the import manager is included in the first post of this thread. Download it if you are running some version other than 1.0.3 of the import manager plugin. This plugin comes prepackaged with the WHMCS migrator, so no need to download that separately if you're upgrading the import manager.

 

If you already have version 1.0.3 of the import manager plugin then simply download the WHMCS migrator as per the instructions.

 

This update will display any errors encountered during the import process. Run it and report back any errors you receive.

Link to comment
Share on other sites

Well, here's the results.  I don't have anything configured on the server to kill long running processes and PHP's max_execution_time is set to over an hour.  This ran for about 10 minutes before stopping.

I'll check the logs on the server and see what I can find...

 

 

The import completed but the following errors ocurred:
importClients: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away on line 124
importContacts: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importTaxes: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importCurrencies: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importInvoices: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importTransactions: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importPackages: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importServices: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importSupportDepartments: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importSupportTickets: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124
importMisc: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124

Edit: I set the mysql wait_timeout to an hour, it was set to 20 minutes.  Also changed some cache settings and the max_allowed_packet size. Trying a new import now.

Link to comment
Share on other sites

Yes, the db is localhost, both blesta and whmcs databases.

The MySQL server did not go away, I confirmed the service uptime.

PHP is configured with a ridiculously high execution time, memory limit and max input time.  MySQL Persistence is enabled and the MySQL Wait timeout is 60 minutes.

The import dies before all clients are imported.  About 90% are imported.

 

I enabled the system benchmark to see how long the import runs for.  I cleared my DB and started again.

Link to comment
Share on other sites

That doesn't tell me anything I don't already know...  The wait_timeout is large enough, the service didn't restart. I'll confirm again after this next import attempt.

 

Import failed again, got more clients imported than last time though.

execution time: 619.885823011 seconds

 
[~]# mysqladmin version
mysqladmin  Ver 8.42 Distrib 5.1.70, for unknown-linux-gnu on x86_64
Copyright © 2000, 2013, Oracle and/or its affiliates. All rights reserved.
 
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
 
Server version          5.1.70-cll
Protocol version        10
Connection              Localhost via UNIX socket
UNIX socket             /var/lib/mysql/mysql.sock
Uptime:                 39 min 23 sec
 
Threads: 2  Questions: 382740  Slow queries: 1  Opens: 194045  Flush tables: 1  Open tables: 1024  Queries per second avg: 161.972
Link to comment
Share on other sites

Ugh, it died again.  No queries in the general query log look bad or broken.  I'm at a loss here.  Mysql is up and did not die.

 

From what I can tell, this is towards the end of the migration where it eventually fails.

 

1 Query     INSERT INTO `contact_numbers` (`contact_id`, `number`, `type`, `location`) VALUES ('[redacted]', '[redacted]', 'phone', 'home')
1 Query     SELECT `id` FROM `modules` WHERE `company_id` = '1' AND `class` = 'universal_module'
1 Query     SELECT COUNT(*) AS `total` FROM ((SELECT * FROM `companies` WHERE `id` = '1')) AS `t_381380570`
1 Query     INSERT INTO `modules` (`company_id`, `class`, `name`, `version`) VALUES ('1', 'universal_module', 'Universal Module', '1.0.10')
1 Query     INSERT INTO `module_rows` (`module_id`) VALUES ('3')
1 Query     INSERT INTO `module_row_meta` (`key`, `value`, `serialized`, `encrypted`, `module_row_id`) VALUES ('name', 'generic_server', '0', '0', '3')
Link to comment
Share on other sites

This is my last attempt today.  I'm getting frustrated.

I did a fresh install of Blesta.  I enabled cPanel and LogicBoxes.  Also enabled Support Manager and Import Manager, then I backed up my database for later restores if this fails.

Import started, crossing fingers.

 

Take a backup of the Mysql database that way you can just delete the tables and re-import the fresh db.

Link to comment
Share on other sites

Have you read the comments on the manual page I linked, above?

 

Have you tried the following:

 

 

The way I managed to rectify the problem was not with the mysql settings, it was actually in php.ini


setting connect_timeout = 300
and default_socket_timeout = 300


they are set to default at 60 seconds and this caused my problem.

 

 

Also, have you looked into increasing mysql's "max_allowed_packet" size?

 

 

P.S. Since the issue is occurring when importing clients, update the migrator (/plugins/import_manager/components/migrators/whmcs/whmcs_migrator.php line 23-36):

 

from:

 

        $actions = array(
            "importStaff", // works
            "importClients", // works
            "importContacts", // works
            "importTaxes", // works
            "importCurrencies", // works
            "importInvoices", // works
            "importTransactions", // works
            "importPackages", // works
            "importServices", // works
            "importSupportDepartments", // works
            "importSupportTickets", // works
            "importMisc" // works
        );

 

to:

 

        $actions = array(
            "importStaff", // works
            "importClients", // works
            #"importContacts", // works
            #"importTaxes", // works
            #"importCurrencies", // works
            #"importInvoices", // works
            #"importTransactions", // works
            #"importPackages", // works
            #"importServices", // works
            #"importSupportDepartments", // works
            #"importSupportTickets", // works
            #"importMisc" // works
        );

 

That way you don't waste time waiting for the rest of the import processes to finish.

Link to comment
Share on other sites

Have you read the comments on the manual page I linked, above?

 

Have you tried the following:

 

 

 

Also, have you looked into increasing mysql's "max_allowed_packet" size?

I'll try the php settings.  I doubled the packet size with no change.  The import didn't grab more data or less for that matter. It dies before all clients are imported and the number of imported clients is very close either way.

Link to comment
Share on other sites

I did a fresh install of Blesta.  I enabled cPanel and LogicBoxes.  Also enabled Support Manager and Import Manager, then I backed up my database for later restores if this fails.

 

You shouldn't have to install any modules or plugins, the importer will do that.. though, it shouldn't be a problem that you did but in our testing we go from a fresh install.

Link to comment
Share on other sites

It failed again.  Cody, I already started before you suggested commenting out the other migration options.  I have commented them out as you suggested.

Paul, I cleared the db and only enabled the import manager and am starting now.

Cody, I did configure those PHP settings and set my mysql allowed_packet_size to 64M

Link to comment
Share on other sites

Another things that might help in terms of debugging:

 

You can get more information about the lost connections by starting mysqld with the --log-warnings=2 option. This logs some of the disconnected errors in the hostname.err file. See http://dev.mysql.com/doc/refman/5.0/en/error-log.html' title="5.2.1. The Error Log">Section 5.2.1, “The Error Log

Link to comment
Share on other sites

This was logged in the hostname.err log.

 

131023 14:07:33 [Warning] Aborted connection 88 to db: 'blesta_whmcs' user: 'blesta_blesta' host: 'localhost' (Got timeout reading communication packets)
 
That was 10 minutes prior to the import failing.  No other errors anywhere.
 

The import completed but the following errors ocurred:

importClients: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away on line 124

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...