Bugfix for negative results and too small results
2000-02-02 => 2001-02-01
> DATEDIF with Y unit: 0 year (returned -1 before fix)
> DATEDIF with YM unit: 11 months (returned -1 before fix)
For functions introduced in Excel 2010 and beyond, Excel saves them
in formulas with the xlfn_ prefix. PhpSpreadsheet does not do this;
as a result, when a spreadsheet so created is opened, the cells
which use the new functions display a #NAME? error.
This the cause of bug report 1246:
https://github.com/PHPOffice/PhpSpreadsheet/issues/1246
This change corrects that problem when the Xlsx writer encounters
a 2010+ formula for a cell or a conditional style. A new class
Writer/Xlsx/Xlfn, with 2 static methods,
is introduced to facilitate this change.
As part of the testing for this, I found some additional problems.
When an unknown function name is used, Excel generates a #NAME? error.
However, when an unknown function is used in PhpSpreadsheet:
- if there are no parameters, it returns #VALUE!, which is wrong
- if there are parameters, it throws an exception, which is horrible
Both of these situations will now return #NAME?
Tests have been added for these situations.
The MODE (and MODE.SNGL) function is not quite in alignment with Excel.
MODE(3, 3, 4, 4) returns 3 in both Excel and PhpSpreadsheet.
However, MODE(4, 3, 3, 4) returns 4 in Excel, but 3 in PhpSpreadsheet.
Both situations will now match Excel's result.
Also, Excel allows its parameters for MODE to be an array,
but PhpSpreadsheet did not; it now will.
There had not been any tests for MODE. Now there are.
The SHEET and SHEETS functions were introduced in Excel 2013,
but were not introduced in PhpSpreadsheet. They are now introduced
as DUMMY functions so that they can be parsed appropriately.
Finally, in common with the "rate" changes for which I am
creating a pull request at the same time as this one:
samples/Basic/13_CalculationCyclicFormulae
PhpUnit started reporting an error like "too much regression".
The test deals with an infinite cyclic formula, and allowed
the calculation engine to run for 100 cycles. The actual number of cycles
seems irrelevant for the purpose of this test. I changed it to 15,
and PhpUnit no longer complains.
There were about 20 skipped tests for RATE and PRICE marked
"This test should be fixed". This change does that by fixing
the code for those functions, validating the existing tests,
and adding new ones. XIRR and XNPV are also substantially changed.
As part of this change, the following functions also have minor changes:
- isValidFrequency
- COUPDAYBS
- COUPNUM (additional tests)
- DB
- DDB
PhpUnit reports 100% coverage for all the changed functions.
Since I was dealing with skipped tests, I also fixed
tests/PhpSpreadsheetTests/Writer/Xlsx/LocaleFloatsTest,
which was being skipped in Windows. I also delete the temporary
file which it creates.
There is now only one remaining test which is skipped -
ODS Reader is not complete enough to run some tests against it.
Unfortunately, that test is too complicated for me to deal with now.
In researching this change, I found several places in the code where special code was added for Gnumeric claiming:
- Gnumeric does not handle free-format string dates
- Gnumeric adds extra options, not available in Excel,
for the frequency parameter for functions such as YIELD
- Gnumeric rounds the results for DB and DDB to 2 decimal places
None of these claims is true, at least not on a recent version
of Gnumeric, and the code which supports these differences is removed.
There did not appear to be any tests targeted for
these supposed properties of Gnumeric.
The PRICE function needed relatively minor changes - mostly
additional tests for invalid input. The main problem with the PRICE
tests is that Excel appears to have a bug. The algorithm is published:
https://support.office.com/en-us/article/price-function-3ea9deac-8dfa-436f-a7c8-17ea02c21b0a
The results that Excel returns for basis codes 2 and 3 appear to be
incorrect in many cases. I have segregated these tests into a
new test PRICE3. The results of these tests agree with the published
algorithm, and to the results for LibreOffice and Gnumeric.
The results returned by Excel do not agree with them.
The tests which remain in the test PRICE all use basis codes other
than 2 or 3, and all agree with Excel, LibreOffice, and Gnumeric.
For the RATE function, there appears to be a problem with how the
secant method was implemented. I studied the implementation of RATE
in Python numpy, and adapted its implementation of secant method.
The results now agree with numpy, and, more important, with Excel.
XIRR, which calls XNPV, permits its dates to be earlier than the
start date, whereas XNPV does not. I dealt with this by renaming
the existing XNPV function to xnpvOrdered, adding a parameter to
indicate whether start date has to be earliest. XNPV calls the new
function with that parameter set to TRUE, and XIRR calls it with
the parameter set to FALSE. Some additional error checking was
added to xnpvOrdered, and also to XIRR. XIRR tests benefited
from increasing the value of FINANCIAL_MAX_ITERATIONS.
Finally, since this change is very test-related:
samples/Basic/13_CalculationCyclicFormulae
PhpUnit started reporting an error like "too much regression".
The test deals with an infinite cyclic formula, and allowed
the calculation engine to run for 100 cycles. The actual number of cycles
seems irrelevant for the purpose of this test. I changed it to 15,
and PhpUnit no longer complains.
Returns #N/A, unless the element searched for is at the end of the array.
The problem is in Calculation.php line 4231:
if (!is_array($functionCall)) {
foreach ($args as &$arg) {
$arg = Functions::flattenSingleValue($arg);
}
unset($arg);
}
I believe this code is intended to handle functions where PhpSpreadsheet just passes
the call on to PHP without implementing the code on its own, e.g. for atan or acos.
In the bug report, the following code fails:
$flat_rate = "=MATCH(6,{4,5,6,2}, 0)";
$sheet->getCell('A1')->setValue($flat_rate);
The expected value is 3, but the actual result is "#N/A".
The reason for this result is that the parser replaces the braces with calls
to the MKMATRIX internal function, whose value for functioncall was:
'self::MKMATRIX'. Since this isn't an array, the flattening code is executed,
and the unintended result occurs. The fix is to change the definition for
functioncall in that case to [__CLASS__, 'mkMatrix'], avoiding the flattening.
However, there is also another part to this bug. The flattening should be
returning the first entry in the array, but is in fact returning the last.
This explains why the bug report specified "unless ... end of the array".
I confirmed that Excel does use the first item in the array rather than the last,
e.g. =atan({1,2,3}) entered into a cell will return atan(1), not atan(3).
The problem here is that flattenSingleValue, which says in its comments that
it is supposed to be returning the first item, uses array_pop rather than array_shift.
I have changed that as well. The same mistake was also present in
Cell.php function getCalculatedValue. The correct behavior can be verified
by entering =minverse({-2.5,1.5;2,-1}) into an Excel cell'
Excel flattens the result ({2,3;4,5}) to 2, and so should PhpSpreadsheet.
Fixes#1271Closes#1332
* Changes to WEEKNUM and YEARFRAC
The optional second parameter for WEEKNUM can take any of 10 values
(1, 2, 11-17, and 21), but currently only 1 and 2 are supported.
This change adds support for the other 8 possibilities.
YEARFRAC in Excel does not require that end date be before start date,
but PhpSpreadsheet was returning an error in that situation.
YEARFRAC third parameter (method) of 1 was not correctly implemented.
I was able to find a description of the algorithm, and documented
that location in the code, and implemented according to that spec.
PHPExcel had a (failing) test to assert the result of
YEARFRAC("1960-12-19", "2008-06-28", 1). This test had been dropped
from PhpSpreadsheet, and is now restored; several new tests have
been added, and verified against Excel.
* Add YEARFRAC Tests
Scrutinizer reported a very mysterious failure with no details.
project.metric_change("scrutinizer.test_coverage", < 0),
without even a link to explain what it is reporting.
It is possible that it was a complaint about code coverage.
If so, I have added some tests which will, I hope, eliminate the problem.
* Make Array Constant
Responding to review from Mark Baker.
* Merge with PR 1362 Bugfix 1161
Travis CI reported problem with Calculation.php (which is not part
of this change).
That was changed in master a few days ago
(delete some unused code).
Perhaps the lack of that change is the problem here.
Merging it manually.
Fix: Return #NUM! if values and dates contain a different number of values
Fix: Return #NUM! if there is not at least one positive cash flow and one negative cash flow
Fix: Return #NUM! if any number in dates precedes the starting date
Fix: Return #NUM! if a result that works cannot be found after max iteration tries
Fix: Correct DocBlocks for XIRR & XNPV
Add: Validate XIRR with unit tests
Closes#1177
In cells with formulas containing conditions like `=IFSUM(A1:A3;"";B1:B3)`
to sum cells from range A1:A3 with empty value in range B1:B3, the function
`Functions::ifCondition()` create in this case the code `=""""` instead of
`=""`, so it didn't work.
Closes#1206
Calculation engine was resolving every function by first resolving its arguments
including IFs, this was causing significant over evaluation when IFs were used
as it meant for every case to be evaluated.
Introduce elements to identify ifs and enable better branch resolution
(pruning). We tag parsed tokens to associate a branch identifier to them.
Closes#844
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* New statistical tests
* Sniffs
* Additional statistical function unit tests
* Additional statistical function unit tests
* Fix case-sensitivity
* Fix HARMEAN code logic
* Unit tests refactored into individual files for all logical functions
Implemented IFNA()
* Fix silly typo
* NOT needs ...args to allow for test when no argument passed
* Codestyle
* Use instance asserts
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* Bessels, and set some date tests to defined/named arguments
* Fix test class naming
* Names arguments for math/trig tests
* Docblock updates
* More engineering function unit test refactorings
* More engineering function unit test refactorings. This time, moving on to the Complex engineering functions
* Fix ImConjugate test
* Fix parseComplex test
* Fix parseComplex test
* More of the complex number function unit tests refactored
* Finish refactoring of the complex number function unit tests
* Newer phpunit assertions
* Add parsecomplex unit test back until we're ready to drop the deprecated function; but as it doesn't use the specified data provider at all, drop reference to that
Sheet titles containing " " or "!" will be quoted in formulas. This commit
fixes the lookup of sheets with this kind of title by trimming the quotes
during the lookup.
Without this any defined range referencing a sheet with " " or "!" in the title
name will be lost when reading the workbook from file.
Fixes#928
Closes 930
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* First pass at moving MathTrig tests into individual test files
* Appeasement to the great goddess PHPCS
* Appeasement to the great goddess PHPCS
* Minor scrutinizer issue resolved
* More refactoring of tests into individual test files fr each math/trig function
* More work on the math/trig test refactoring, plus a bit of tidyup of date/time tests as well
* Fix test
* Fix docblock in test
* Finish refactoring Math/Trig tests into separate files
* Fix SubTotal Test
* Import additional classes for SubTotal test
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* Separate out date/time tests into individual tests
* Need to update the version of phpunit at some point to deal with the new assertions and deprecated assertions
* Appease the CS Gods
* More refactoring of Date/Time tests
* Replace self assertions with instance assertions (looking forward to upgrading phpunit)
* Finish refactoring of date/time tests as individual tests
* Test for DateTimeInterface rather than for DateTime
* A few strict comparisons
* Fix to test names
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* Further statistical tests
* Additional BINOMDIST tests
Extract boolean and string checking for Average and Count functions into separate methods
* Appease the great god PHPCS
* Yet more appeasement of the great god PHPCS
* Beginning to get really cheesed off with PHPCS, pulling me up over full stops in comments... I want to see this completed before going to bed; but it's nearly half past one in the morning, and phpcs has been pulling me up over trivialities for the past f***ing hour
* And a spurious line
* Further work on statistical tests
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* Adjusted logic for COUNT() function to handle differences in EXCEL, GNUMERIC and OPENOFFICE modes for cells and for literal values
* Fix case-sensitivity in filenames
* Appeasing Codesniffer
* Resolve COUNTA() differences between cell values and literals
* Style fixes
* Start refactoring statistical function tests into individual tests rather than having a single, giant test for all statistical functions.... first step toward doing this for all tests
* More refactoring into separate tests
If all functions have their own individual test files, it should be a lot easier to identify which functions aren't covered by tests yet
* Missing last lines in files
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* More trend function unit tests
* Yet more trend function unit tests
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* More statistical tests
* Further statistical tests
* Unit tests for some of the trend functions
* resolve scrutiniser objections
* Fix order of @return types :-(
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* Additional unit tests for average functions, and fix to AVERAGEIF() function if third argument is passed
* Update change log
* Stricter typed comparisons in AVERAGEIF() conditions
* Unit tests for BETADIST() and BETAINV()
* New Unit Tests for COUPNUM()
* COUPNUM should not return zero when settlement is in the last period
* Additional tests and fixes for COUPNCD() and COUPPCD() functions
* Highlight VLOOKUP bug with a new test
* Remove useless statements + fix VLOOKUP bug
Please not that we have still inconsistencies in Excel (See LOOKUP and
VLOOKUP.php test files)
* Base strtolower on our StringHelper in LookupRef
Alban Duval