Skip to content

Commit

Permalink
git-remote-mediawiki: fix encoding issue for UTF-8 media files
Browse files Browse the repository at this point in the history
When a media file contains valid UTF-8, git-remote-mediawiki tried to be
too clever about the encoding, and the call to utf8::downgrade() on the
downloaded content was failing with

  Wide character in subroutine entry at git-remote-mediawiki line 583.

Instead, use $response->decode() to apply decoding linked to the
Content-Encoding: header, and return the content without attempting any
charset decoding.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
  • Loading branch information
Matthieu Moy authored and Junio C Hamano committed Apr 23, 2014
1 parent 1c4ea83 commit 9742fb7
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 1 deletion.
7 changes: 6 additions & 1 deletion contrib/mw-to-git/git-remote-mediawiki.perl
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,12 @@ sub download_mw_mediafile {

my $response = $mediawiki->{ua}->get($download_url);
if ($response->code == HTTP_CODE_OK) {
return $response->decoded_content;
# It is tempting to return
# $response->decoded_content({charset => "none"}), but
# when doing so, utf8::downgrade($content) fails with
# "Wide character in subroutine entry".
$response->decode();
return $response->content();
} else {
print {*STDERR} "Error downloading mediafile from :\n";
print {*STDERR} "URL: ${download_url}\n";
Expand Down
19 changes: 19 additions & 0 deletions contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,25 @@ test_expect_success 'git clone works on previously created wiki with media files
test_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt
'

test_expect_success 'git push can upload media (File:) files containing valid UTF-8' '
wiki_reset &&
git clone mediawiki::'"$WIKI_URL"' mw_dir &&
(
cd mw_dir &&
"$PERL_PATH" -e "print STDOUT \"UTF-8 content: éèàéê€.\";" >Bar.txt &&
git add Bar.txt &&
git commit -m "add a text file with UTF-8 content" &&
git push
)
'

test_expect_success 'git clone works on previously created wiki with media files containing valid UTF-8' '
test_when_finished "rm -rf mw_dir mw_dir_clone" &&
git clone -c remote.origin.mediaimport=true \
mediawiki::'"$WIKI_URL"' mw_dir_clone &&
test_cmp mw_dir_clone/Bar.txt mw_dir/Bar.txt
'

test_expect_success 'git push & pull work with locally renamed media files' '
wiki_reset &&
git clone mediawiki::'"$WIKI_URL"' mw_dir &&
Expand Down

0 comments on commit 9742fb7

Please sign in to comment.