If row or column dimensions are accessed, then HTML writer would
still generate lots of empty cells, to show nothing at all. This
now ignore row and column dimensions to only output cell that
actually exists (even if those cells are empty).
Fixes#1235Close#1537
* Xls Writer - Correct Timestamp Bug, Improve Coverage
I believe that Xls Writer is 100% covered now.
The Xls Writer sets its timestamp incorrectly. The problem is actually
in Shared/Ole::localDateToOLE, which converts its timestamp using
gmmktime; mktime is correct. If I save a file at 3:00 p.m. in San Francisco,
this bug means the time is actually recorded as 3:00 p.m. UTC.
A consequence of this is that if you use Phpspreadsheet to read the
file and save it as a new Xls, the creation timestamp goes further
and further back in time with each generation (or further forward
if east of Greenwich). One of the tests added confirms that
the creation timestamp is consistent with the start and end times
of the test.
The major change in coverage is adding tests to save GIF and BMP
images, which aren't supported in Xls, but are converted to PNG
in the PhpSpreadsheet code.
* Fix For #1509
User expected no CSV enclosures after $writer->setEnclosure(''),
which had been changed to be consistent with $reader->setEnclosure('').
Writer will now omit enclosures after code above; no change to Reader.
Tests have been added for this condition.
* Add Option to Write CSV Enclosure Only When Required
Allowing the user to specify no enclosure when writing a CSV can lead to
a situation where PhpSpreadsheet (likewise Excel) will not read the
resulting file as intended, e.g. if any cell contains a delimiter character.
This is demonstrated in new test TestBadReread.
No existing setting will rectify this situation.
A better choice would be to add an option to write the enclosure
only when it is needed, which is what Excel does. The RFC4180 spec at
https://tools.ietf.org/html/rfc4180
states when it is needed - when the cell contains the delimiter,
or the enclosure, or a newline.
New test TestGoodReread demonstrates that the file is read as intended.
The documentation has been updated to describe the new function,
and to change the write example where the enclosure is set to null.
* Scrutinizer Suggestions
3 minor changes, all in tests.
This problem is that ZipStream, in contrast to ZipArchive,
is saving 2 files with the same path. I have opened an issue with
ZipStream, who agree that this appears to be a bug.
For the case in question, PhpSpreadsheet is attempting to save
a file with the same path twice (and unexpectedly succeeding)
because of a clone operation. This fix attempts to rectify the problem
by keeping track of all the paths being saved in the zip file,
and not attempting to save any duplicate paths.
The problem case attempted to save printersettings1.bin twice,
but there are other possible exposures, e.g. by cloning a sheet
with a drawing.The new test cases clone an existing sample which
has both printer settings and drawings.
Replace default gridlines with different style. Usable in PDF
as well as HTML.
Documentation mentioned use of setUseBOM with Html, but that method
does not exist, and there is no real reason to support it.
Removed it from documentation.
We give users the ability to edit Html/Pdf, but it's a little cumbersome
to use the edited Html for an Html file, and difficult to use it
for a Pdf. I believe we could make it fairly painless in both cases
by allowing the user to set a callback to edit the generated Html.
This can be accomplished with fewer than a dozen lines of very simple code.
I think this would be easier than grabbing the Html in pieces,
editing it, and reassembling it. I think it would also be simpler
than an alternative I considered, namely the addition of a new method
(e.g. saveEditedHtml) to each of the Html and Pdf writers.
One edit that users might like to make when editing html is to add
fallback fonts, something that is not currently available in
PhpSpreadsheet, and might be difficult to add. A natural extension to
that idea would be the use of webfonts, something which is guaranteed
difficult to add. See samples/Basic/17b_Html for an example of this.
None of the PDF writers support webfonts yet. That doesn't mean they
won't do so in future, but, for now, samples/Pdf/21a_Pdf is a prosaic
example of something you could do with this callback. In fact, this
opens the door to letting the user replace the entire body with data
of their choosing, effectively allowing PhpSpreadsheet (where you can
set things like paper size and orientation) to be used as a front-end to
the Pdf processor without the user having to be be overly familiar with
the vagaries of the PDF processor. I think this is actually a pretty
nice idea. YMMV. See samples/Basic/21b_Pdf for an example.
No code changes. The tests in all of these scripts write to at least
one temporary file, which is then read and not used again. The file
should be deleted to avoid filling up the disk system.
This test changes directory then performs an assertion.
No problem if the assertion succeeds. I was a little concerned about
what would happen if the assertion fails, leaving us in the
new directory. So I have changed test to use setUp/tearDown
to ensure that we end up where we started.
There are a number of situations where HTML write was producing
HTML which could not be validated. These include:
- inconsistent use of backslash terminating META, IMG, and COL tags
- @page style tags in body rather than header. Aside from being
non-standard, HTML Reader treats those as spreadsheet data.
- <div style="page-break-before:always" />, a construct which is
usually better handled through css anyhow.
- no alt tag for images (drawings and charts)
Other problems:
- Windows file names not handled correctly for images
- Memory drawings not handled in extendRowsForChartsAndImages
- No handling of different values for showing gridlines
for screen and print
- Mpdf and Dompdf do not require the use of inline css.
Tcpdf remains a holdout in the use of this inferior approach.
- no need to chunk base64 encoding of embedded images
- support for colors in number format was buggy (html tags
run through htmlspecialchars)
Code has been refactored when practical to reduce the number of
very large functions.
Coverage is now 100% for the entire HTML Writer module,
from 75% lines and 39% methods beforehand.
All functions dealing only with charts
are bypassed for coverage because the version of Jpgraph available in
Composer is not suitable for PHP7. The code will, nevertheless,
run successfully, but with warning messages. I have confirmed that
the code is entirely covered, without warnings, when the current
version of Jpgraph is used in lieu of the one available in Composer.
I will be glad to revisit this when the Jpgraph problem is resolved.
Directory PhpSpreadsheetTests/Writer/Html was created to house
the new tests. It seemed logical to move HtmlCommentsTest to
the new directory from PhpSpreadsheetTests/Functional.
A function to generate all the HTML is useful, especially for testing,
but also in lieu of the multiple other generate* functions. I have
added and documented generateHTMLAll.
The documentation for the generate* functions (a) produces invalid html,
(b) produces html which cannot be handled correctly by HTML reader,
and (c) even if those were correct, does not actually affect
the display of the spreadsheet. The documentation has been replaced
by a valid, and more instructive, example.
The (undocumented) useEmbeddedCss property, and the functions
to test and set it are no longer needed. Rather than breaking
existing code by deleting them, I marked the functions deprecated.
This change borrows a change to LocaleFloatsTest from
pull request 1456, submitted a little over a week before this one.
## Improve NumberFormat Support
First phase of this change included correcting NumberFormat handling
in HTML Writer. Certain complex formats could not be handled without
changes to Style/NumberFormat, and I did not wish to combine those changes.
Once the original change had been pushed, I took this part of it back up.
HTML Writer can now handle conditions in formats like:
[Blue][>=3000.5]$#,##0.00;[Red][<0]$#,##0.00;$#,##0.00
In testing, I discovered several errors and omissions
in handling of some other formats.
These are now corrected, and tests added.
There were about 20 skipped tests for RATE and PRICE marked
"This test should be fixed". This change does that by fixing
the code for those functions, validating the existing tests,
and adding new ones. XIRR and XNPV are also substantially changed.
As part of this change, the following functions also have minor changes:
- isValidFrequency
- COUPDAYBS
- COUPNUM (additional tests)
- DB
- DDB
PhpUnit reports 100% coverage for all the changed functions.
Since I was dealing with skipped tests, I also fixed
tests/PhpSpreadsheetTests/Writer/Xlsx/LocaleFloatsTest,
which was being skipped in Windows. I also delete the temporary
file which it creates.
There is now only one remaining test which is skipped -
ODS Reader is not complete enough to run some tests against it.
Unfortunately, that test is too complicated for me to deal with now.
In researching this change, I found several places in the code where special code was added for Gnumeric claiming:
- Gnumeric does not handle free-format string dates
- Gnumeric adds extra options, not available in Excel,
for the frequency parameter for functions such as YIELD
- Gnumeric rounds the results for DB and DDB to 2 decimal places
None of these claims is true, at least not on a recent version
of Gnumeric, and the code which supports these differences is removed.
There did not appear to be any tests targeted for
these supposed properties of Gnumeric.
The PRICE function needed relatively minor changes - mostly
additional tests for invalid input. The main problem with the PRICE
tests is that Excel appears to have a bug. The algorithm is published:
https://support.office.com/en-us/article/price-function-3ea9deac-8dfa-436f-a7c8-17ea02c21b0a
The results that Excel returns for basis codes 2 and 3 appear to be
incorrect in many cases. I have segregated these tests into a
new test PRICE3. The results of these tests agree with the published
algorithm, and to the results for LibreOffice and Gnumeric.
The results returned by Excel do not agree with them.
The tests which remain in the test PRICE all use basis codes other
than 2 or 3, and all agree with Excel, LibreOffice, and Gnumeric.
For the RATE function, there appears to be a problem with how the
secant method was implemented. I studied the implementation of RATE
in Python numpy, and adapted its implementation of secant method.
The results now agree with numpy, and, more important, with Excel.
XIRR, which calls XNPV, permits its dates to be earlier than the
start date, whereas XNPV does not. I dealt with this by renaming
the existing XNPV function to xnpvOrdered, adding a parameter to
indicate whether start date has to be earliest. XNPV calls the new
function with that parameter set to TRUE, and XIRR calls it with
the parameter set to FALSE. Some additional error checking was
added to xnpvOrdered, and also to XIRR. XIRR tests benefited
from increasing the value of FINANCIAL_MAX_ITERATIONS.
Finally, since this change is very test-related:
samples/Basic/13_CalculationCyclicFormulae
PhpUnit started reporting an error like "too much regression".
The test deals with an infinite cyclic formula, and allowed
the calculation engine to run for 100 cycles. The actual number of cycles
seems irrelevant for the purpose of this test. I changed it to 15,
and PhpUnit no longer complains.
I believe that both CSV Reader and Writer are 100% covered now.
There were some errors uncovered during development.
The reader specifically permits encodings other than UTF-8 to be used.
However, fgetcsv will not properly handle other encodings.
I tried replacing it with fgets/iconv/strgetcsv, but that could not
handle line breaks within a cell, even for UTF-8.
This is, I'm sure, a very rare use case.
I eventually handled it by using php://memory to hold the translated
file contents for non-UTF8. There were no tests for this situation,
and now there are (probably too many).
"Contiguous" read was not handle correctly. There is a file
in samples which uses it. It was designed to read a large sheet,
and split it into three. The first sheet was corrrect, but the
second and third were almost entirely empty. This has been corrected,
and the sample code was adapted into a formal test with assertions
to confirm that it works as designed.
I made a minor documentation change. Unlike HTML, where you never
need a BOM because you can declare the encoding in the file,
a CSV with non-ASCII characters must explicitly include a BOM
for Excel to handle it correctly. This was explained in the Reading CSV
section, but was glossed over in the Writing CSV section, which I
have updated.
* Handle Error in Formula Processing Better for Xls
When there is an error writing a formula to an Xls file,
which seems to happen when a defined name is part of a formula,
the cell is currently left blank. A better result would be to
write the calculated value of the formula.
* Making Changes Suggested in Review
Per comment from Mark Baker:
1. Made return codes from writeFormula into constants.
2. Skipped redundant call to getCellValue when possible.
3. Added support for bool type, adding additional tests
for bool and string.
Per comment from PowerKiki:
1. Used standardized convention for assigning file name in test.
Since before this change, save would throw Exception, I kept
the unlink for the file in tearDown.
* Initial unit test for locale floats
This will require potential modification of the TravisCI environment to support other locales
* var_dump to check output on TravisCI
* Fix assertions for double/float and with/without line reference
* Style in unit test
Prior to 1.10, all numeric values where read as floats. In 1.10
numeric values are read using 0 + x, which relies on PHP type
juggling rules. As a result, float(0.0) is written as string('0'),
then read back as int(0). This fix causes the writer to retain the
the decimal for float values such that a reader can differentiate
floats from ints.
Closes#1262
This will let users read a file that contains data that are not properly
supported and write them back to a new file untouched.
- load workbookProtection attributes
- save loaded pageSetup[r:id]
- save loaded sheet's AlternateContent
- save loaded unparsed VmlDrawings
- save loaded drawing files `rId`
- save loaded draw's AlternateContent
- save loaded control properties
- save loaded printer settings
- save loaded unparsed override content types (for ctrlProp, ...)
Closes#435
Because even if it doesn't make a difference in practice, it is
technically more correct to call static methods statically. It
also better advertise that those methods can be used from any context.
Adrien Crivelli