FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » FUDforum Installation Issues » Import problem at fud30_msg (import fails or hangs and dies)
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
Import problem at fud30_msg [message #167339] Fri, 08 June 2012 21:44 Go to next message
ShineOn is currently offline  ShineOn   United States
Messages: 53
Registered: July 2011
Karma: 1
Member
add to buddy list
ignore all messages by this user
FUDforum 3.0.3 moving from one host to another.

Have copy of FUD working on new server, trying to restore latest dump from old server to new server.

Import fails at table fud30_msg, either with the classic can't allocate more memory error that was fixed a long time ago, or if I change the memory limit in admimport to -1 it just hangs and dies at fud30_msg.

I tried changing the mysql my.cnf to the huge configuration thinking maybe that would give it enough juice. I don't know what else to try. Not getting error messages logged anywhere I can find.

Old host uses cPanel, new host uses DirectAdmin.
Re: Import problem at fud30_msg [message #167342 is a reply to message #167339] Sat, 09 June 2012 03:17 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3681
Registered: December 2004
Karma: 19
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
"Just hangs" means it's probably busy. Monitor your system to see what's going on. Look at processor utilization and do a "SHOW PROCESSLIST" on MySQL to show active queries.

"Just dies" means an error have be logged somewhere. Have you looked for errors in both your web server (probably apache) and database log files?
Re: Import problem at fud30_msg [message #167357 is a reply to message #167342] Sat, 09 June 2012 16:23 Go to previous messageGo to next message
ShineOn is currently offline  ShineOn   United States
Messages: 53
Registered: July 2011
Karma: 1
Member
add to buddy list
ignore all messages by this user
Frank, thanks, as usual, for your prompt reply.

Process list shows no mysql activity. The process is being killed by the kernel when I set memory_limit to -1 in admimport.

The line I have been messing with is the @ini_set line, from
/* main */
	@ini_set('memory_limit', '256M');
	@set_time_limit(0);


With admimport memory_limit set to -1, this is what shows at the tail end of the import screen in adm panel:
Process table fud30_mime:
...43 rows loaded.
Process table fud30_mod:
...143 rows loaded.
Process table fud30_msg:
The relevant section of the dmesg log:
Out of memory: kill process 14258 (php-cgi) score 363611 or a child
Killed process 14258 (php-cgi) vsz:2908888kB, anon-rss:864564kB, file-rss:16kB


Again, this is with memory_limit set to -1 instead of 256M in admimport. With the stock admimport memory_limit of 256M, I get this in adm panel:
Process table fud30_mime:
...43 rows loaded.
Process table fud30_mod:
...143 rows loaded.
Process table fud30_msg:
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 523800 bytes) in /home/<user>/<domain>_files/FUDforum/include/theme/default/db.inc on line 168 
and in the forum apache error log:
PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 523800 bytes) in /home/<usedr>/<domain>_files/FUDforum/include/theme/default/db.inc on line 168, referer: http://<url>/adm/admimport.php?rid=42&SQ=e222280852faf75391ed5282969bc414


Note: if I set the memory_limit in admimport to 512M, the error that's thrown is identical, except it has a higher memory value and says "... db.inc on line 53, ..." instead of line 168

System info:
FUDforum version: 	3.0.3
PHP version: 	5.3.13
PHP built on: 	Linux <server name> 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64
Database type: 	mysql (mysqli)
Database version: 	5.5.16
Web server: 	Apache/2
Web Server load: 	0.03
Web server to PHP interface: 	cgi-fcgi
WebServer User/Group: 	<user> / <group> (503/505)
Relevant PHP settings: 	
Safe mode: 	OFF
Open basedir: 	/home/<user>/:/tmp/
Display errors: 	ON
File uploads: 	ON
Maximum file upload size: 	32M
Magic quotes: 	ON
Output buffering: 	Yes
Disabled functions: 	exec, system, passthru, shell_exec, escapeshellarg, escapeshellcmd, proc_close, proc_open, dl, popen, show_source
PSpell support: 	No
Zlib support: 	Yes


Please note I am still tweaking settings on the new host server, so if there are any settings I should change immediately, please advise.

Also, the old host is no longer available, so all I have to work with is the backup file I am trying to restore, so please don't ask me to redo the dump - I can't.
Re: Import problem at fud30_msg [message #167358 is a reply to message #167357] Sat, 09 June 2012 16:25 Go to previous messageGo to next message
ShineOn is currently offline  ShineOn   United States
Messages: 53
Registered: July 2011
Karma: 1
Member
add to buddy list
ignore all messages by this user
I just noticed - magic quotes is on - shouldn't that be off?

I turned it off, but still get the unable to allocate message...

[Updated on: Sat, 09 June 2012 16:33]

Report message to a moderator

Re: Import problem at fud30_msg [message #167359 is a reply to message #167358] Sun, 10 June 2012 02:58 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3681
Registered: December 2004
Karma: 19
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Aanhaling:
Out of memory: kill process 14258 (php-cgi) score 363611 or a child
Killed process 14258 (php-cgi) vsz:2908888kB, anon-rss:864564kB, file-rss:16kB


Your kernel is killing PHP because it uses too much memory. Changing PHP settings won't help. Not sure how to resolve it, but adding memory or swap space may resolve it. Also check if you haven't allocated too much memory to some processes, like MySQL for example.
Re: Import problem at fud30_msg [message #167379 is a reply to message #167359] Mon, 11 June 2012 21:05 Go to previous messageGo to next message
ShineOn is currently offline  ShineOn   United States
Messages: 53
Registered: July 2011
Karma: 1
Member
add to buddy list
ignore all messages by this user
I trimmed down all processes as much as possible. My VPS has 1GB RAM and 2GB swap, and after clean reboot, starting out with 0 swap used, running the import results in swap filling up completely and OOM killing php. Something else must be wrong. Is there a way to bypass the msg table import and run that separately?

[Updated on: Mon, 11 June 2012 22:00]

Report message to a moderator

Re: Import problem at fud30_msg [message #167382 is a reply to message #167379] Wed, 13 June 2012 01:06 Go to previous messageGo to next message
ShineOn is currently offline  ShineOn   United States
Messages: 53
Registered: July 2011
Karma: 1
Member
add to buddy list
ignore all messages by this user
Here's the answer:

I found that simple PMA imports of largish tables (12,000+ entries) also failed, with a php timeout. I changed Apache's global timeout (was set to 30 sec, bumped to 3 min.) and PHP's timeout (was 5 sec, set to 180 sec) - overkill, yes, but that still wasn't the total answer, as I was still getting oom issues.

What fixed it was changing the php memory limit from 256M to 64M. The import completed successfully as soon as I made that change.
Re: Import problem at fud30_msg [message #167393 is a reply to message #167382] Thu, 14 June 2012 02:00 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3681
Registered: December 2004
Karma: 19
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Great! Thanks for the update.
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: update to 3.0 not working
Next Topic: Synchronize to mailing list not sending anything to list
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Oct 17 01:21:41 EDT 2018

Total time taken to generate the page: 0.00714 seconds