FRH Dave Posted October 23, 2013 Report Share Posted October 23, 2013 Just attempted an import, and no package data came through. The packages themselves were created, but were not assigned to customers. What info do you need? EDIT: Some came through, some did not. I'm researching to see what the difference is. EDIT 2: The only customer to get a package associated correctly is my test account, user #1 in WHMCS. And then only one package (out of maybe a dozen) came through. I do see many packages created successfully in Blesta, but any package in WHMCS that did not have a non-zero quantity is showing in Blesta with 0 quantity. If the package quantity in WHMCS was either "unlimited" or "0", it's now "0" in Blesta. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Does the importer import email activity? The table tblemails to be exact. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 I just ran the importer on my rather large database. It worked this time from localhost instead of over the network. Items in red did not import, items in green were confirmed imported, items in orange cannot be confirmed because the do not apply. Staff (including login passwords) Client Groups Clients (excluding login passwords) Credit Card Payment Accounts Custom Client Fields and Values Client Notes Contacts - Imported as clients, this can be an issue... Tax Rules Currencies Invoices Recurring Invoices Transactions Client Credits Support Departments Support Tickets (excluding attachments) Support Predefined Response Categories Support Predefined Responses Sent Emails The following Settings: Mail Delivery Type SMTP Host, User, Password, Port, and Security Type Invoice Days Before Renewal Invoice Reminder Settings Auto Debit Days Before Due Suspend Services After Due Calendar Events Products (become packages) Registrars (become modules/module rows) Servers (become modules/module rows) Services Domains (become services) Quote Link to comment Share on other sites More sharing options...
Paul Posted October 23, 2013 Report Share Posted October 23, 2013 It should import all of those things, chances are it is failing at some point. We'll need to identify the error you're getting so that we can try to fix it. Cody should be able to provide information to enable debugging on the importer. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Please do, I would really like to get this working! Quote Link to comment Share on other sites More sharing options...
FRH Dave Posted October 23, 2013 Report Share Posted October 23, 2013 No users above user #1 have their package data imported. Maybe this is related to the fact that all the packages that were "unlimited" in WHMCS now have 0 quantity in Blesta? Like Blesta set the limit to 0, then said "no inventory left, can't add that package". Quote Link to comment Share on other sites More sharing options...
Cody Posted October 23, 2013 Author Report Share Posted October 23, 2013 The latest version of the import manager is included in the first post of this thread. Download it if you are running some version other than 1.0.3 of the import manager plugin. This plugin comes prepackaged with the WHMCS migrator, so no need to download that separately if you're upgrading the import manager. If you already have version 1.0.3 of the import manager plugin then simply download the WHMCS migrator as per the instructions. This update will display any errors encountered during the import process. Run it and report back any errors you receive. Michael 1 Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Upgraded to 1.0.3 and am starting a new import after reverting my database. Will report the results shortly (5-10 minutes). Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Well, here's the results. I don't have anything configured on the server to kill long running processes and PHP's max_execution_time is set to over an hour. This ran for about 10 minutes before stopping. I'll check the logs on the server and see what I can find... The import completed but the following errors ocurred: importClients: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away on line 124 importContacts: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importTaxes: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importCurrencies: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importInvoices: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importTransactions: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importPackages: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importServices: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importSupportDepartments: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importSupportTickets: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 importMisc: SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens on line 124 Edit: I set the mysql wait_timeout to an hour, it was set to 20 minutes. Also changed some cache settings and the max_allowed_packet size. Trying a new import now. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 It failed again, same error. Quote Link to comment Share on other sites More sharing options...
Paul Posted October 23, 2013 Report Share Posted October 23, 2013 That's a tough one, looks like MySQL is crashing or times out. Is it running on localhost? Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Yes, the db is localhost, both blesta and whmcs databases. The MySQL server did not go away, I confirmed the service uptime. PHP is configured with a ridiculously high execution time, memory limit and max input time. MySQL Persistence is enabled and the MySQL Wait timeout is 60 minutes. The import dies before all clients are imported. About 90% are imported. I enabled the system benchmark to see how long the import runs for. I cleared my DB and started again. Quote Link to comment Share on other sites More sharing options...
Cody Posted October 23, 2013 Author Report Share Posted October 23, 2013 See the MySQL manual for reason why the MySQL may "go away". Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 That doesn't tell me anything I don't already know... The wait_timeout is large enough, the service didn't restart. I'll confirm again after this next import attempt. Import failed again, got more clients imported than last time though. execution time: 619.885823011 seconds [~]# mysqladmin version mysqladmin Ver 8.42 Distrib 5.1.70, for unknown-linux-gnu on x86_64 Copyright © 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Server version 5.1.70-cll Protocol version 10 Connection Localhost via UNIX socket UNIX socket /var/lib/mysql/mysql.sock Uptime: 39 min 23 sec Threads: 2 Questions: 382740 Slow queries: 1 Opens: 194045 Flush tables: 1 Open tables: 1024 Queries per second avg: 161.972 Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Ugh, it died again. No queries in the general query log look bad or broken. I'm at a loss here. Mysql is up and did not die. From what I can tell, this is towards the end of the migration where it eventually fails. 1 Query INSERT INTO `contact_numbers` (`contact_id`, `number`, `type`, `location`) VALUES ('[redacted]', '[redacted]', 'phone', 'home') 1 Query SELECT `id` FROM `modules` WHERE `company_id` = '1' AND `class` = 'universal_module' 1 Query SELECT COUNT(*) AS `total` FROM ((SELECT * FROM `companies` WHERE `id` = '1')) AS `t_381380570` 1 Query INSERT INTO `modules` (`company_id`, `class`, `name`, `version`) VALUES ('1', 'universal_module', 'Universal Module', '1.0.10') 1 Query INSERT INTO `module_rows` (`module_id`) VALUES ('3') 1 Query INSERT INTO `module_row_meta` (`key`, `value`, `serialized`, `encrypted`, `module_row_id`) VALUES ('name', 'generic_server', '0', '0', '3') Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 This is my last attempt today. I'm getting frustrated. I did a fresh install of Blesta. I enabled cPanel and LogicBoxes. Also enabled Support Manager and Import Manager, then I backed up my database for later restores if this fails. Import started, crossing fingers. Biscuit1001 1 Quote Link to comment Share on other sites More sharing options...
Michael Posted October 23, 2013 Report Share Posted October 23, 2013 This is my last attempt today. I'm getting frustrated. I did a fresh install of Blesta. I enabled cPanel and LogicBoxes. Also enabled Support Manager and Import Manager, then I backed up my database for later restores if this fails. Import started, crossing fingers. Take a backup of the Mysql database that way you can just delete the tables and re-import the fresh db. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Take a backup of the Mysql database that way you can just delete the tables and re-import the fresh db. That's pretty much what I said I did... Took a backup of the database for later restores. Import failed again, no usable errors. Quote Link to comment Share on other sites More sharing options...
Cody Posted October 23, 2013 Author Report Share Posted October 23, 2013 Have you read the comments on the manual page I linked, above? Have you tried the following: The way I managed to rectify the problem was not with the mysql settings, it was actually in php.ini setting connect_timeout = 300and default_socket_timeout = 300 they are set to default at 60 seconds and this caused my problem. Also, have you looked into increasing mysql's "max_allowed_packet" size? P.S. Since the issue is occurring when importing clients, update the migrator (/plugins/import_manager/components/migrators/whmcs/whmcs_migrator.php line 23-36): from: $actions = array( "importStaff", // works "importClients", // works "importContacts", // works "importTaxes", // works "importCurrencies", // works "importInvoices", // works "importTransactions", // works "importPackages", // works "importServices", // works "importSupportDepartments", // works "importSupportTickets", // works "importMisc" // works ); to: $actions = array( "importStaff", // works "importClients", // works #"importContacts", // works #"importTaxes", // works #"importCurrencies", // works #"importInvoices", // works #"importTransactions", // works #"importPackages", // works #"importServices", // works #"importSupportDepartments", // works #"importSupportTickets", // works #"importMisc" // works ); That way you don't waste time waiting for the rest of the import processes to finish. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 Have you read the comments on the manual page I linked, above? Have you tried the following: Also, have you looked into increasing mysql's "max_allowed_packet" size? I'll try the php settings. I doubled the packet size with no change. The import didn't grab more data or less for that matter. It dies before all clients are imported and the number of imported clients is very close either way. Quote Link to comment Share on other sites More sharing options...
Cody Posted October 23, 2013 Author Report Share Posted October 23, 2013 Make sure you update the migrator file as well before your next attempt. Quote Link to comment Share on other sites More sharing options...
Paul Posted October 23, 2013 Report Share Posted October 23, 2013 I did a fresh install of Blesta. I enabled cPanel and LogicBoxes. Also enabled Support Manager and Import Manager, then I backed up my database for later restores if this fails. You shouldn't have to install any modules or plugins, the importer will do that.. though, it shouldn't be a problem that you did but in our testing we go from a fresh install. Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 It failed again. Cody, I already started before you suggested commenting out the other migration options. I have commented them out as you suggested. Paul, I cleared the db and only enabled the import manager and am starting now. Cody, I did configure those PHP settings and set my mysql allowed_packet_size to 64M Quote Link to comment Share on other sites More sharing options...
Cody Posted October 23, 2013 Author Report Share Posted October 23, 2013 Another things that might help in terms of debugging: You can get more information about the lost connections by starting mysqld with the --log-warnings=2 option. This logs some of the disconnected errors in the hostname.err file. See http://dev.mysql.com/doc/refman/5.0/en/error-log.html' title="5.2.1. The Error Log">Section 5.2.1, “The Error Log Quote Link to comment Share on other sites More sharing options...
tsiedsma Posted October 23, 2013 Report Share Posted October 23, 2013 This was logged in the hostname.err log. 131023 14:07:33 [Warning] Aborted connection 88 to db: 'blesta_whmcs' user: 'blesta_blesta' host: 'localhost' (Got timeout reading communication packets) That was 10 minutes prior to the import failing. No other errors anywhere. The import completed but the following errors ocurred: importClients: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away on line 124 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.