Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTML or direct PDF output #8

Closed
asantoni opened this issue Nov 22, 2019 · 22 comments · Fixed by #53 · May be fixed by #42
Closed

HTML or direct PDF output #8

asantoni opened this issue Nov 22, 2019 · 22 comments · Fixed by #53 · May be fixed by #42

Comments

@asantoni
Copy link

I was really excited to try this out, but then really bummed by only having XML or LaTeX output - I can't read either. I spent like 20 minutes trying to set up a Tex distribution and going through the loop of compiling, failing, installing another Tex style, and repeating until I gave up because of some weird error about the fontawesome style.

My point is, it would be awesome if flan output some easily human readable format directly, without the need for some external dependencies to process it (no Tex, no XML parser, etc.). For now, I'm sorta stuck squinting at these XML files....
@bonedaddy
Copy link

bonedaddy commented Nov 22, 2019 •
edited
Loading

sudo apt install texlive-full on ubuntu did it for me, but it required a 2.8GB download to install the full thing. I had the same problems as you and installing texlive full worked.

You can then convert latex document to pdf using a tool pdflatex

@akhepcat
Copy link

texi2pdf works on the resulting TeX file - but for some reason the escaping of various chars ( '_' , '#', etc) wasn't occuring correctly on my pull, so a manual fixup was required before it would convert correctly.

Seems the script has the sed lines to do it, so i don't know why it was failing.

@asantoni
Copy link
Author

texi2pdf worked for me. Having to install 3GB of Tex packages sure takes the shine off flan being distributed with a tiny, efficient Alpine Docker image....

@Suhail
Copy link

Suhail commented Nov 22, 2019 •
edited
Loading

Agreed - it was really difficult to get this to work with .tex -- lots of dependencies involved. I tried for 30 min before giving up.

@moosbaue
Copy link

moosbaue commented Nov 22, 2019 •
edited
Loading

on debian even install texlive-full did not fully do the trick:
I get this error:
! Undefined control sequence. <argument> \n l.57 ...gument to urllib.request.urlopen with \r\n (specifically in the path...
nonstopmode did let it finish (just enter r at the question mark prompt)

pdflatex -interaction=nonstopmode report... does compile with lot of warnings

Tried something with lots of vulnerabilities
! Misplaced \cr. <inserted text> \cr l.121 \end{enumerate} That makes 100 errors; please try again.

cumbersome

If I had a wish: HTML

@jrobinson52
Copy link

I also cannot get the latex to compile properly (openSUsE 15.1) I get lots of the 'Undefined control sequence' errors as well

@ghost
Copy link

ghost commented Nov 26, 2019

Getting latex setup can be...challenging at the best of times. If flan must output to latex, please lets put a viable latex install in the docker image, and do the conversion there - it'll save a lot of frustration out here.

@akhepcat
Copy link

So, since I was able to get the TeX to PDF working mostly cleanly, here's what TeX/PDF packages I have in stalled in my Ubuntu 18.04.3 (bionic) system:

# dpkg -l | grep -i tex | grep -vi text | grep -i '^i'   
ii  docbook2x                                 0.8.8-16                                                 amd64        Converts DocBook/XML documents into man pages and TeXinfo
ii  fonts-lyx                                 2.2.4-0ubuntu0.18.04.1                                   all          TrueType versions of some TeX fonts used by LyX
ii  fonts-texgyre                             20160520-1                                               all          OpenType fonts based on URW Fonts
ii  latex-beamer                              3.24-1                                                   all          LaTeX class to produce presentations
ii  latex-xcolor                              2.11-1.1                                                 all          Easy driver-independent TeX class for color
ii  libkpathsea5                              2009-11ubuntu2                                           amd64        TeX Live: path search library for TeX (runtime part)
ii  libkpathsea6:amd64                        2017.20170613.44572-8ubuntu0.1                           amd64        TeX Live: path search library for TeX (runtime part)
ii  libpod-latex-perl                         0.61-2                                                   all          module to convert Pod data to formatted LaTeX
ii  libptexenc1:amd64                         2017.20170613.44572-8ubuntu0.1                           amd64        TeX Live: pTeX encoding library
ii  libsynctex1:amd64                         2017.20170613.44572-8ubuntu0.1                           amd64        TeX Live: SyncTeX parser library
ii  libtexlua52:amd64                         2017.20170613.44572-8ubuntu0.1                           amd64        TeX Live: Lua 5.2, modified for use with LuaTeX
ii  libtexluajit2:amd64                       2017.20170613.44572-8ubuntu0.1                           amd64        TeX Live: LuaJIT, modified for use with LuaJITTeX
ii  pgf                                       2.10-1                                                   all          TeX Portable Graphic Format
ii  preview-latex-style                       11.91-1ubuntu1                                           all          extraction of elements from LaTeX documents as graphics
ii  tex-common                                6.09                                                     all          common infrastructure for building and installing TeX
ii  tex-gyre                                  20160520-1                                               all          scalable PostScript and OpenType fonts based on URW Fonts
ii  texi2html                                 1.82+dfsg1-5                                             all          Convert Texinfo files to HTML
ii  texinfo                                   6.5.0.dfsg.1-2                                           amd64        Documentation system for on-line information and printed output
ii  texlive-base                              2017.20180305-1                                          all          TeX Live: Essential programs and files
ii  texlive-binaries                          2017.20170613.44572-8ubuntu0.1                           amd64        Binaries for TeX Live
ii  texlive-fonts-extra                       2017.20180305-2                                          all          TeX Live: Additional fonts
ii  texlive-fonts-extra-links                 2017.20180305-2                                          all          TeX Live:
ii  texlive-fonts-recommended                 2017.20180305-1                                          all          TeX Live: Recommended fonts
ii  texlive-formats-extra                     2017.20180305-2                                          all          TeX Live: Additional formats
ii  texlive-generic-extra                     2017.20180305-1                                          all          TeX Live: transitional dummy package
ii  texlive-htmlxml                           2017.20180305-1                                          all          TeX Live: transitional dummy package
ii  texlive-latex-base                        2017.20180305-1                                          all          TeX Live: LaTeX fundamental packages
ii  texlive-latex-extra                       2017.20180305-2                                          all          TeX Live: LaTeX additional packages
ii  texlive-latex-recommended                 2017.20180305-1                                          all          TeX Live: LaTeX recommended packages
ii  texlive-pictures                          2017.20180305-1                                          all          TeX Live: Graphics, pictures, diagrams
ii  texlive-plain-generic                     2017.20180305-2                                          all          TeX Live: Plain (La)TeX packages
ii  texlive-xetex                             2017.20180305-1                                          all          TeX Live: XeTeX and packages
ii  tipa                                      2:1.3-20                                                 all          system for processing phonetic symbols in LaTeX

and PDF:

# dpkg -l | grep -i pdf  | grep -i '^i'   
ii  asciidoc-base                             8.6.10-2                                                 all          Minimal version of asciidoc not suitable for pdf
ii  ghostscript                               9.26~dfsg+0-0ubuntu0.18.04.12                            amd64        interpreter for the PostScript language and for PDF
ii  libgs9:amd64                              9.26~dfsg+0-0ubuntu0.18.04.12                            amd64        interpreter for the PostScript language and for PDF - Library
ii  libgs9-common                             9.26~dfsg+0-0ubuntu0.18.04.12                            all          interpreter for the PostScript language and for PDF - common files
ii  libpoppler13                              0.16.7-2ubuntu2                                          amd64        PDF rendering library
ii  libpoppler19:amd64                        0.18.4-1ubuntu3.1                                        amd64        PDF rendering library
ii  libpoppler44:amd64                        0.24.5-2ubuntu4.4                                        amd64        PDF rendering library
ii  libpoppler73:amd64                        0.62.0-2ubuntu2.10                                       amd64        PDF rendering library
ii  libqpdf13:amd64                           5.1.1-1                                                  amd64        runtime library for PDF transformation/inspection software
ii  libqpdf21:amd64                           8.0.2-3                                                  amd64        runtime library for PDF transformation/inspection software
ii  php-tcpdf                                 6.2.13+dfsg-1ubuntu1                                     all          PHP class for generating PDF files on-the-fly
ii  poppler-data                              0.4.8-2                                                  all          encoding data for the poppler PDF rendering library
ii  poppler-utils                             0.62.0-2ubuntu2.10                                       amd64        PDF utilities (based on Poppler)
ii  python3-reportlab                         3.4.0-3build1                                            all          ReportLab library to create PDF documents using Python3
ii  qpdf                                      8.0.2-3                                                  amd64        tools for transforming and inspecting PDF files
ii  xpdf                                      3.04-7                                                   amd64        Portable Document Format (PDF) reader

@moosbaue
Copy link

Picking up on docker self containement: I started to develop another way of getting a pdf report as outlined by Chris Moffitt (hxxps://pbpython.com/author/chris-moffitt.html) here hxxps://pbpython.com/pdf-reports.html.
The idea is creating the report in html using jinja2 and output pdf by feeding the html into WeasyPrint. Could be the easier way.
And hopefully at the end no dependencies outside of docker. My challenge here will be CSS though, as I have limited knowledge and experience there.

@mdaocus
Copy link

mdaocus commented Nov 27, 2019

Latex on CentOS? Pure frustration ...

@catcherxuefeng
Copy link

cloudconvert -f pdf report_2019.11.28-02.29.tex

@christian-korneck christian-korneck mentioned this issue Nov 28, 2019
Closed
@christian-korneck
Copy link

christian-korneck commented Nov 28, 2019 •
edited
Loading

I've had the same problem and have added pdf conversion + upload (in the container, not the host system) in pr #42 . It increases image build time and container size a lot, but getting a pdf is really essential for my use case.

@christian-korneck christian-korneck mentioned this issue Nov 28, 2019
Open
@moosbaue
Copy link

Picking up on docker self containement: I started to develop another way of getting a pdf report as outlined by Chris Moffitt (hxxps://pbpython.com/author/chris-moffitt.html) here hxxps://pbpython.com/pdf-reports.html.
The idea is creating the report in html using jinja2 and output pdf by feeding the html into WeasyPrint. Could be the easier way.
And hopefully at the end no dependencies outside of docker. My challenge here will be CSS though, as I have limited knowledge and experience there.

#41
means I can stop my adventure !

@moosbaue
Copy link

or even not:;
same issues as with "! Undefined control sequence. ....."

@christian-korneck
Copy link

can you share the invalid latex document that you have?

@moosbaue
Copy link

moosbaue commented Nov 29, 2019 •
edited
Loading

obviously the issues ly in the way the description text of the vulnerability is writen:
one example is "...does not properly handle a '\0...." which throws an error
the other on was "....gument to urllib.request.urlopen with \r\n....

So I helped myself out with adding
sed -i 's/\r\n//g' $root_dir$report_file
sed -i 's/\0//g' $root_dir$report_file
just above
latexmk -f -pdf -outdir=$root_dir/reports $root_dir$report_file #make pdf

Bad thing you cannot control what peaple put in the description of vulnerability. I do not know Latex: Ist there an equivalent for the html pre tag as this might be able to solve this challenge

And maybe we can do all this sanatizing in get_description(vuln, type) in file output_report.py by using
return cve_json["description"]["description_data"][0]["value"].decode("utf8","ignore")
or even easier one line above
cve_json = json.loads(urllib.urlopen(url).read().decode("utf-8","ignore"))
worth a try - will do it on another day

@christian-korneck
Copy link

christian-korneck commented Nov 29, 2019 •
edited
Loading

@moosbaue can you maybe share a full .tex file that fails to convert to pdf? I can't reproduce your issue (I'm getting nice looking pdf files with/without detected vulnerabilities with pr #42 , tried it against multiple servers).

(Maybe share it as gist?)

@moosbaue
Copy link

here is the invalid dcument
report_2019.11.30-16.12.tex.txt
it has two issues

  1. \r\n in some descriprtions which can be removed safely
    sed -i 's/\\r\\n//g' $root_dir$report_file
  2. "...a '\0' character....". I figure how to double escape it: '\0' -> '\\0'
    sed -i 's/'\\0''/'\\\0''/g' $root_dir$report_file
    tried to to exand it to all alphanumeric character but failed

@christian-korneck
Copy link

@moosbaue thanks, I can see what the problem is. Could you maybe also share the .xml file, so that I can test a potential fix against it?

@moosbaue
Copy link

moosbaue commented Dec 1, 2019

here they are:
192.168.178.28.xml.txt
192.168.178.33.xml.txt

@marius-udubasa
Copy link

I was able to have a quick pdf quality result by just pasting the tex file content (you can import the file too) into the online tool from here https://www.overleaf.com/ and then using Recompile and pdf download

@Horaddrim Horaddrim mentioned this issue Dec 9, 2019
Closed
@shadowscatcher shadowscatcher mentioned this issue Mar 8, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
11 participants
@Suhail @moosbaue @asantoni @marius-udubasa @akhepcat @christian-korneck @mdaocus @bonedaddy @catcherxuefeng @jrobinson52 and others