| Re: DOS newlines (CR/LF) to Unix format [message #171731] |
Wed, 19 January 2011 11:24  |
Kim Andr Aker
Messages: 17
Registered: September 2010
Karma: 0
|
Junior Member |
|
|
På Wed, 19 Jan 2011 11:59:48 +0100, skrev Bjarne Jensen
<bjarne(dot)b(dot)jensen(at)gmail(dot)com>:
> I found this snippet on internet:
>
> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>
> It works fine on the commandline so I wrote like this in a php-script:
>
> - - -
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
> }
>
> - - -
>
>
> But absolutely nothing happens!
>
> Why not?
It's probably either one of two reasons, or a combination of the two.
One, your glob() function may not return any items.
Two, your system may have disabled the use of the exec(), passthru()
and/or system() functions due to security. In any case, running a command
for this in PHP is not needed, just use the built-in PHP functions instead.
Assuming you use PHP 5, and that your use of the glob() function is
correct:
$unix = ".unix";
foreach (glob($usedir."[12_]*") as $filename) {
// doing this in a single operation
file_put_contents($filename.$unix, str_replace("\r", "",
file_get_contents($filename)));
}
--
Kim André Akerø
- kimandre(at)NOSPAMbetadome(dot)com
(remove NOSPAM to contact me directly)
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171732 is a reply to message #171731] |
Wed, 19 January 2011 11:25  |
Luuk
Messages: 329
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 19-01-11 11:59, Bjarne Jensen wrote:
> I found this snippet on internet:
>
> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>
> It works fine on the commandline so I wrote like this in a php-script:
>
> - - -
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
the single quotes around your AWK-script are missing...
> }
>
> - - -
>
>
> But absolutely nothing happens!
there might be an error on your log..
>
> Why not?
>
>
> /Bjarne
you also could try 'dos2unix'
see: man dos2unix
or: http://linux.die.net/man/1/dos2unix
--
Luuk
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171733 is a reply to message #171731] |
Wed, 19 January 2011 11:25 |
alvaro.NOSPAMTHANX
Messages: 277
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
El 19/01/2011 11:59, Bjarne Jensen escribió/wrote:
> I found this snippet on internet:
>
> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>
> It works fine on the commandline so I wrote like this in a php-script:
>
> - - -
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
> }
>
> - - -
>
>
> But absolutely nothing happens!
>
> Why not?
Read this:
http://www.php.net/manual/en/language.types.string.php
The important bit is that in PHP single quoted strings are different
from double quoted strings (it's exactly the same as in bash). Single
quoted strings are stored as-is, so 'foo $bar' contains exactly that:
foo $bar.
You can check the exact command you are running by using the ancient
debug technique of printing stuff with an echo statement:
foreach (glob($usedir."[12_]*") as $filename) {
echo 'awk { sub("/\r$", ""); print } $filename > $filename.$unix');
}
Now, injecting unescaped random stuff in a command line can be very
dangerous. You should always use escapeshellarg()!
Last but not least... I assume you are using PHP code to launch bash
code because you don't have a full script in either language to do the
full task, am I correct? Your approach is weird and probably
inefficient. Google for "xargs awk", you'll probably find a 100% bash
example.
--
-- http://alvaro.es - Álvaro G. Vicario - Burgos, Spain
-- Mi sitio sobre programación web: http://borrame.com
-- Mi web de humor satinado: http://www.demogracia.com
--
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171734 is a reply to message #171731] |
Wed, 19 January 2011 11:33 |
Thomas 'PointedEars'
Messages: 701
Registered: October 2010
Karma: 0
|
Senior Member |
|
|
Bjarne Jensen wrote:
> I found this snippet on internet:
>
> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>
> It works fine on the commandline so I wrote like this in a php-script:
>
> - - -
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
> }
>
> - - -
>
> But absolutely nothing happens!
Untrue.
> Why not?
Assuming that you are allowed to exec(): You should have copy-pasted; the
shell command is not the same. In essence, it is an awk(1) invocation
error: Unquoted `{' and `}' are for Brace Expansion in a POSIX-compatible
shell.
But you do not need a shell or awk(1) to accomplish what you want (although
dos2unix(1)/fromdos(1) is arguably faster than awk(1)), PHP has e.g.
fread(), preg_replace(), and fwrite().
PointedEars
--
Use any version of Microsoft Frontpage to create your site.
(This won't prevent people from viewing your source, but no one
will want to steal it.)
-- from <http://www.vortex-webdesign.com/help/hidesource.htm> (404-comp.)
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171735 is a reply to message #171731] |
Wed, 19 January 2011 12:47 |
Michael Fesser
Messages: 215
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
.oO(Bjarne Jensen)
> I found this snippet on internet:
>
> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>
> It works fine on the commandline so I wrote like this in a php-script:
>
> - - -
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
> }
>
> - - -
>
>
> But absolutely nothing happens!
>
> Why not?
Besides the other replies - may I ask why you need this? I consider line
ending conversion a relic from the 1980s. Today only very few and very
old and very badly written software should have a problem with different
line endings.
Micha
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171741 is a reply to message #171732] |
Thu, 20 January 2011 03:15 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-19 12:25, Luuk wrote:
> On 19-01-11 11:59, Bjarne Jensen wrote:
>> I found this snippet on internet:
>>
>> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
>> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>>
>> It works fine on the commandline so I wrote like this in a php-script:
>>
>> - - -
>>
>> $unix = ".unix";
>>
>> foreach (glob($usedir."[12_]*") as $filename) {
>> exec('awk { sub("/\r$", ""); print } $filename> $filename.$unix');
>
> the single quotes around your AWK-script are missing...
Now corrected bot still no joy.
>> }
>>
>> - - -
>>
>>
>> But absolutely nothing happens!
>
> there might be an error on your log..
error_log now activated
>
>>
>> Why not?
>>
>>
>> /Bjarne
>
> you also could try 'dos2unix'
also no joy.
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171742 is a reply to message #171733] |
Thu, 20 January 2011 03:20 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-19 12:25, "Álvaro G. Vicario" wrote:
<zipped>
>
> Now, injecting unescaped random stuff in a command line can be very
> dangerous. You should always use escapeshellarg()!
Point taken.
> Last but not least... I assume you are using PHP code to launch bash
> code because you don't have a full script in either language to do the
> full task, am I correct? Your approach is weird and probably
> inefficient. Google for "xargs awk", you'll probably find a 100% bash
> example.
The bash version works fine but I need to create something that can be
used by someone even less competent in computers/scripting than I am as
I'm not allways present.
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171743 is a reply to message #171734] |
Thu, 20 January 2011 03:26 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-19 12:33, Thomas 'PointedEars' Lahn wrote:
<zipped>
> Assuming that you are allowed to exec(): You should have copy-pasted; the
> shell command is not the same. In essence, it is an awk(1) invocation
> error: Unquoted `{' and `}' are for Brace Expansion in a POSIX-compatible
> shell.
I can do like exec("whoami") fine so I assume that I should be permitted
to awk, dos2unix etc. ?
> But you do not need a shell or awk(1) to accomplish what you want (although
> dos2unix(1)/fromdos(1) is arguably faster than awk(1)), PHP has e.g.
> fread(), preg_replace(), and fwrite().
Speed is no issue but preg_replace() is beginning to look beautiful...
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171744 is a reply to message #171735] |
Thu, 20 January 2011 03:32 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-19 13:47, Michael Fesser wrote:
<zipped>
> Besides the other replies - may I ask why you need this? I consider line
> ending conversion a relic from the 1980s. Today only very few and very
> old and very badly written software should have a problem with different
> line endings.
It is about CSV data entry into a database. I'm just trying to eliminate
possible errors to ease use of different databases.
MySQL has a 'lines ending with..." option but others may not.
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171889 is a reply to message #171743] |
Thu, 20 January 2011 10:26 |
Thomas 'PointedEars'
Messages: 701
Registered: October 2010
Karma: 0
|
Senior Member |
|
|
Bjarne Jensen wrote:
> On 2011-01-19 12:33, Thomas 'PointedEars' Lahn wrote:
>> Assuming that you are allowed to exec(): You should have copy-pasted; the
>> shell command is not the same. In essence, it is an awk(1) invocation
>> error: Unquoted `{' and `}' are for Brace Expansion in a POSIX-compatible
>> shell.
>
> I can do like exec("whoami") fine so I assume that I should be permitted
> to awk, dos2unix etc. ?
Maybe. It depends on the permissions that the user running php(.exe) has
for those files.
The main issue is that you need to quote the argument to awk(1) as it was
described (single-quotes to avoid shell expansion).
PointedEars
--
realism: HTML 4.01 Strict
evangelism: XHTML 1.0 Strict
madness: XHTML 1.1 as application/xhtml+xml
-- Bjoern Hoehrmann
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171892 is a reply to message #171889] |
Thu, 20 January 2011 12:03 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-20 11:26, Thomas 'PointedEars' Lahn wrote:
> Maybe. It depends on the permissions that the user running php(.exe) has
> for those files.
The user running the script is 'www-data'.
But I'm lost in space when it comes to determining what permissions
'www-data' has or would need to exec other programs.
> The main issue is that you need to quote the argument to awk(1) as it was
> described (single-quotes to avoid shell expansion).
I'm trying to sort that one out.
/Bjarne
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171894 is a reply to message #171892] |
Thu, 20 January 2011 13:17 |
Luuk
Messages: 329
Registered: September 2010
Karma: 0
|
Senior Member |
|
|
On 20-01-11 13:03, Bjarne Jensen wrote:
> On 2011-01-20 11:26, Thomas 'PointedEars' Lahn wrote:
>
>> Maybe. It depends on the permissions that the user running php(.exe) has
>> for those files.
>
> The user running the script is 'www-data'.
> But I'm lost in space when it comes to determining what permissions
> 'www-data' has or would need to exec other programs.
>
>
>> The main issue is that you need to quote the argument to awk(1) as it was
>> described (single-quotes to avoid shell expansion).
>
> I'm trying to sort that one out.
>
>
> /Bjarne
If you have this PHP-file:
<?php
exec('awk --version >awk.txt', $result, $retun_var);
print_r($result);
print "Return_var: $return_var <br>";
exec("awk '{ print $1 }' awk.txt", $two, $return_var);
print_r($two);
print "Return_var: $return_var <br>";
?>
The output should be:
Array ( ) Return_var:
Array ( [0] => GNU [1] => Copyright [2] => [3] => This [4] => it [5] =>
the [6] => (at [7] => [8] => This [9] => but [10] => MERCHANTABILITY
[11] => GNU [12] => [13] => You [14] => along ) Return_var: 0
(at least if your awk-version is "GNU Awk 3.1.6" ;)
If awk cannot be executed, the output will be:
Array ( ) Return_var:
Array ( ) Return_var: 2
--
Luuk
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171898 is a reply to message #171894] |
Thu, 20 January 2011 15:45 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
On 2011-01-20 14:17, Luuk wrote:
> If you have this PHP-file:
> <?php
> exec('awk --version>awk.txt', $result, $retun_var);
> print_r($result);
> print "Return_var: $return_var<br>";
>
> exec("awk '{ print $1 }' awk.txt", $two, $return_var);
> print_r($two);
> print "Return_var: $return_var<br>";
> ?>
>
>
> The output should be:
> Array ( ) Return_var:
> Array ( [0] => GNU [1] => Copyright [2] => [3] => This [4] => it [5] =>
> the [6] => (at [7] => [8] => This [9] => but [10] => MERCHANTABILITY
> [11] => GNU [12] => [13] => You [14] => along ) Return_var: 0
>
> (at least if your awk-version is "GNU Awk 3.1.6" ;)
>
> If awk cannot be executed, the output will be:
> Array ( ) Return_var:
> Array ( ) Return_var: 2
Array
(
)
Return_var: 0
Array
(
[0] => GNU
[1] => Copyright
[2] =>
[3] => This
[4] => it
[5] => the
[6] => (at
[7] =>
[8] => This
[9] => but
[10] => MERCHANTABILITY
[11] => GNU
[12] =>
[13] => You
[14] => along
)
Return_var: 0
So - it is there but....
|
|
|
|
| Re: DOS newlines (CR/LF) to Unix format [message #171904 is a reply to message #171731] |
Thu, 20 January 2011 18:58 |
Bjarne Jensen
Messages: 9
Registered: January 2011
Karma: 0
|
Junior Member |
|
|
Hej Kim,
Mange tak - det funker!
Mvh Bjarne
- - -
On 2011-01-19 12:24, Kim André Akerø wrote:
> På Wed, 19 Jan 2011 11:59:48 +0100, skrev Bjarne Jensen
> <bjarne(dot)b(dot)jensen(at)gmail(dot)com>:
>
>> I found this snippet on internet:
>>
>> # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
>> awk '{sub(/\r$/,"");print}' # assumes EACH line ends with Ctrl-M
>>
>> It works fine on the commandline so I wrote like this in a php-script:
>>
>> - - -
>>
>> $unix = ".unix";
>>
>> foreach (glob($usedir."[12_]*") as $filename) {
>> exec('awk { sub("/\r$", ""); print } $filename > $filename.$unix');
>> }
>>
>> - - -
>>
>>
>> But absolutely nothing happens!
>>
>> Why not?
>
> It's probably either one of two reasons, or a combination of the two.
>
> One, your glob() function may not return any items.
>
> Two, your system may have disabled the use of the exec(), passthru()
> and/or system() functions due to security. In any case, running a
> command for this in PHP is not needed, just use the built-in PHP
> functions instead.
>
> Assuming you use PHP 5, and that your use of the glob() function is
> correct:
>
> $unix = ".unix";
>
> foreach (glob($usedir."[12_]*") as $filename) {
> // doing this in a single operation
> file_put_contents($filename.$unix, str_replace("\r", "",
> file_get_contents($filename)));
> }
>
>
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
]