Friday, November 21, 2014

Use Windows' text to speech API from your Perl script

Jonathan Stowe commented on Facebook:

I hate say in any computer language that doesn't actually make audible speaking sounds.

Well, you can use Windows' text to speech API from Perl:

#!/usr/bin/env perl

use 5.020; # Why not?!
use strict;
use warnings;

use Win32::OLE;
$Win32::OLE::Warn = 3;

my $speaker = Win32::OLE->new('SAPI.spvoice');

while (my $line = <DATA>) {
    next unless $line =~ /\S/;
    $speaker->Speak( $line );
}

__DATA__
Flexible & Powerful

That's why we love Perl 5

Visit Perl.org

Now, if you want to replace the builtin say, I am afraid you are stuck. But, even with the use 5.020; line, you could add a no feature 'say';, and define your own say function.

Additionally, the speech API understands some XML directives.

Perl6 UTF-8 output in cmd.exe on Windows 8.1

Back in May, starting with UTF-8 output from Perl and C programs in cmd.exe on Windows 8, I wrote a few posts summarizing my bewilderment with extra bytes appearing in UTF-8 output from perl when cmd.exe codepage was set to UTF-8 via chcp 65001.

The same problem still exists with the perl 5.20.1 I built recently.

So, I decided to see what perl6, using MoarVM, can give me.

In the same cmd.exe Window, I typed:

C:\Temp> perl6 -e "Buf.new(0xce, 0xb1, 0xce, 0xb2, 0xce, 0xb3, 0x31).decode('UTF-8').say"
αβγ1

or, in a script:

use v6;
'αβγ1'.say;

which gave me the output:

αβγ1

It may not seem like much, but remember that the Perl script:

use utf8;
use strict;
use warnings;
use warnings qw(FATAL utf8);

binmode STDOUT, ':utf8';

print 'αβγ1', "\n";

outputs

αβγ1
1

Let's see now:

.say for (
    "Hava karlı",
    "Bu iş kârlı",
    "İstanbul",
    "Yağ yağ yağmur");
Hava karlı
Bu iş kârlı
İstanbul
Yağ yağ yağmur

And, then,

say "karlı" eq "kârlı";
False

which means perl6 understands the difference between snowy and profitable.

Of course, so does perl5 ... but, when it comes to printing, it has issues:

use 5.020;
use utf8;
binmode STDOUT, ":utf8";

print for "karlı", "kârlı";

gives the output:

karlıkârlılı�

Monday, November 17, 2014

TDD is all well and good, but who's testing the tests?

While working on my patch to Module::Install, I happened to do another cpanoutdated | cpanm -v for my shiny new Perl 5.20.1 build. This pulled in Test::More, with another one of those file path interpolation into regex pattern problems:

Here are the relevant parts:

Unrecognized escape \M passed through in regex; marked by <-- HERE in m/The new sub is 'MyModernTester::__ANON__' defined in t\Behavior\M <-- HERE onkeyPatching_diag.t

Got: The new sub is 'MyModernTester::__ANON__' defined in t\Behavior\MonkeyPatching_diag.t around line 19.

Expected: (?^:The new sub is 'MyModernTester::__ANON__' defined in t\Behavior\MonkeyPatching_diag.t around line 19)

This is not the only instance of this type of test failure.

Please, please, use quotemeta.

Here is the pull request. It consists solely of replacing qr{ … $file … } with qr{ … \Q$file\E … }.

Friday, November 14, 2014

Who put an extra CR in my CRLF? Fun with PerlIO layers

So, playing with my shiny new toy while waiting for remote SAS jobs to finish, I ended up trying to install PerlIO::via::gzip which pulls in PerlIO::Util. PerlIO::Util failed some of its tests, specifically, one relating to PerlIO::tee.

Some of the failures involved an extra 0d before a 0d0a

These failures were perplexing to me, but, at least CPAN testers showed me that I was not alone.

On the other hand, looking at that list, there are clearly Perl installations on Windows where this does not happen. I have been unable to figure out the underlying cause, but I suspect it is at least somewhat related to the series of puzzled posts I made about UTF-8 output from perl in a cmd.exe Window.

I went ahead, and force installed PerlIO::Util just 'cause I am lazy, and I wanted to use PerlIO::via::gzip. Here is a short script to start with. We are printing to scalars:

#!/usr/bin/env perl

use 5.020;
use strict;
use warnings;

use PerlIO::Util;

open my $f, '>:tee', \(my ($x, $y))
    or die "tee open: $!";

binmode $f, ':crlf' or die "binmode: $!";

print $f "\n";

close $f;

say hexdump($_) for $x, $y;

# Thanks for the tip in the comments
sub hexdump { sprintf('%*v02x', ' ', $_[0]) }

And, the output is:

 t.pl
0d 0a
0d 0a

or

 t.pl | xxd
0000000: 3064 2030 610d 0a30 6420 3061 0d0a       0d 0a..0d 0a..

But, if I do this:

#!/usr/bin/env perl

use 5.020;
use strict;
use warnings;

use PerlIO::Util;

open my $f, '>:tee', 'x', 'y'
    or die "tee open: $!";

binmode $f, ':crlf' or die "binmode: $!";

print $f "abc\n";

close $f;

I get:

 xxd x
0000000: 6162 630d 0d0a                           abc...

 xxd y
0000000: 6162 630d 0d0a                           abc...

That is, it looks like the "\n" above gets translated to CRLF, and then another layer translates the last LF to CRLF again.

It seems to my untrained eye that whatever is happening is probably happening within this function in PerlIO-Util.xs:

if(tab && tab->Open){
  f = tab->Open(aTHX_ tab, layers, i,  mode,
    fd, imode, perm, f, narg, args);

  /* apply 'upper' layers
     e.g. [ :unix :perlio :utf8 :creat ]
                          ~~~~~        
  */

  if(f && ++i < n){
   if(PerlIO_apply_layera(aTHX_ f, mode, layers, i, n) != 0){
    PerlIO_close(f);
    f = NULL;
   }
  }

 }

A quick inspection seems to verify this. Here is the list of layers before the application of the :crlf layer:

---               
- unix            
- ~               
- - CANWRITE      
  - OPEN          
  - TRUNCATE      
---               
- crlf            
- ~               
- - CANWRITE      
  - FASTGETS      
  - CRLF          
  - TRUNCATE      
---               
- tee             
- y               
- - CANWRITE      
  - FASTGETS      
  - CRLF          
  - TRUNCATE

and, here is the list of layers after the binmode $f, ':crlf':

---            
- unix         
- ~            
- - CANWRITE   
  - OPEN       
  - TRUNCATE   
---            
- crlf         
- ~            
- - CANWRITE   
  - FASTGETS   
  - CRLF       
  - TRUNCATE   
---            
- tee          
- y            
- - CANWRITE   
  - FASTGETS   
  - CRLF       
  - TRUNCATE   
---            
- crlf         
- ~            
- - CANWRITE   
  - FASTGETS   
  - CRLF       
  - TRUNCATE

I have demonstrated in the past that I don't necessarily understand PerlIO layers very well. But, perldoc PerlIO says:

:crlf

A layer that implements DOS/Windows like CRLF line endings. On read converts pairs of CR,LF to a single "\n" newline character. On write converts each "\n" to a CR,LF pair. Note that this layer will silently refuse to be pushed on top of itself. (emphasis mine)

Any ideas?

Thursday, November 13, 2014

Compile Vim and OpenSSL with Visual Studio 2013 Community Edition

They both build fine. For OpenSSL, follow the instructions at Building OpenSSL 1.0.1g on 64-bit Windows Pro 8.1 with Windows SDK 7.1. For Vim, refer to Building and Installing gVim on Windows 8.1 Pro.

Since 7.4.393, you can build GVim with DirectX support on Windows Vista and later. Subjectively, things look much better to me.

If you run into problems with Vim, make sure Windows SDK 7.1 is installed, and add its include directory (and only the include) to your header search path (for Win32.mak):

set INCLUDE=%INCLUDE%;C:\Program Files\Microsoft SDKs\Windows\v7.1\Include

The correct include path to point to the SDK included with the VS2013CE would be:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Include

Again, this is just for the Win32.mak file, so it doesn't make a huge difference, but, still.

After building OpenSSL, you can do:

set OPENSSL_PREFIX=c:\your-openssl-top-dir
cpanm LWP::Protocol::https

That reminds me, I should support OPENSSL_PREFIX in Crypt::SSLeay as well.

On my system, a test in t\io-socket-ip.t fails for a reason I cannot figure out. It may have to do with peculiarities of my system, so I force installed IO::Socket::SSL.

Wednesday, November 12, 2014

You've gotta quotemeta!

Seriously, if you are going to interpolate a filename into a string that is to be used as a regex pattern, you must use quotemeta. That is, just surround the filename with \Q and \E. It doesn't take much, but it does prevent unforced errors.

So, after building my shiny new Perl 5.20.1 with the Community Edition of Visual Studio 2013, I did a:

C:\Users\user> cpanm Moo Template Mojolicious

After a few minutes, I looked, and saw that a bunch of dependencies had failed to build.

First to catch my attention was Test::LeakTrace — odd because the test files giving the problems were written more than five years ago, but I didn't run into this problem with any of the previous times I installed the aforementioned modules. … A-ha! Template Toolkit started checking for leaks.

What are these tests that are failing in Test::LeakTrace? Glad you asked. Here is one:

like __FILE__, qr/$si->[1]/, 'state info'
    or diag(Dumper \@info);

And the others are here, here, here, and here.

Yeah, sure, I am going to put together a patch … but … seriously?!

PS: Rather curiously, Test::Fatal also fails tests. See if you can spot the reason:

#   Failed test '$TODO works'
#   at t\todo.t line 110.
# STDOUT is:
# not ok 1 - succeeded # TODO unimplemented
# #   Failed (TODO) test 'succeeded'
# #   at t\todo.t line 108.
# #          got: '0'
# #     expected: '1'
# ok 2 - no exceptions # TODO unimplemented
# ok 3 - level 1 # TODO unimplemented
#
# not:
# not ok 1 - succeeded # TODO unimplemented
# #   Failed (TODO) test 'succeeded'
# #   at t/todo.t line 108.
# #          got: '0'
# #     expected: '1'
# ok 2 - no exceptions # TODO unimplemented
# ok 3 - level 1 # TODO unimplemented
#
# as expected

Now, this file also has not changed in a long time. Presumably, the way the filename is formatted for the failure message changed recently, but I am not in the mood for a long, uncertain git bisect session.

Of course, the solution to this problem has nothing to do with quotemeta, but I just could not bring myself to put together another blog post.

64-bit Perl 5.20.1 with Visual Studio 2013 Community Edition on Windows 8.1

Microsoft have made Visual Studio 2013 Community Edition available. You can use it for free under what I consider to be fairly liberal provisions:

Q: Who can use Visual Studio Community?

A: Here’s how individual developers can use Visual Studio Community:

  • Any individual developer can use Visual Studio Community to create their own free or paid apps.
  • Here’s how Visual Studio Community can be used in organizations:

    An unlimited number of users within an organization can use Visual Studio Community for the following scenarios: in a classroom learning environment, for academic research, or for contributing to open source projects.

    For all other usage scenarios: In non-enterprise organizations, up to 5 users can use Visual Studio Community. In enterprise organizations (meaning those with >250 PCs or > $1MM in annual revenue), no use is permitted beyond the open source, academic research, and classroom learning environment scenarios described above.

So, I went ahead, and downloaded it. The dialog told me I needed 10 GB for the installation. So, I waited some. At first, I omitted the phone SDK … I am assuming I can download that later if I do want to give it a shot.

My Windows 8 laptop is now close to a decade old. It now has 4 GB memory installed, but can only see 3 GB by design. VS 2013 CE claims it needs 1 GB memory. We'll see how we command line builds work with what I have.

Unfortunately, following the 40 minute install process was done, something happened to the network connection. After a quick restart, I found 19 important updates waiting for me in the Windows Update queue. While waiting for those to install, I configured a ConEmu tab for VS 2013 x64 Native Tools Command Prompt:

I then switched to perl-5.20.1\win32, and edited the Makefile

CCTYPE = MSVC120
…
CCHOME = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64
CCINCDIR = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include
CCLIBDIR = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\amd64

to make sure the linker found the 64-bit libraries.

I also set:

OPTIMIZE = -O1 /favor:INTEL64 -MD -Zi -DNDEBUG

because, after all, I have an Intel CPU in this thing.

Then, I typed nmake and pressed Enter.

It is too bad that there are soo many conversion from size_t to I32 warnings here. I should look into this at some point. For now, these custom built versions are only being used on my own laptop to see if things work, modules build etc, so it is not a priority.

nmake test ended with a few test failures I had not seen before:

I can come back to those later.

In the mean time, I downloaded and installed cpanm and cpan-outdated, and issued a:

C:\Users\user> cpan-outdated | cpanm -v

Next, it's going to be the latest version of OpenSSL, and a whole bunch of other Perl modules that are currently installed alongside the other perl 5.20.1 built with the WinSDK 7.1 compiler.

For now, the updated modules are compiling.