[1] Rail: train wheels run on two rails made of steel. Sections of rail vary in length. The ends of the sections generally abut one another. Due to expansion and contraction during different seasons, the gap between the rails may contract and expand. This gap gives rise to the characteristic “clickety-clack”
sound of trains.
[2] Rail base: the rail base is the base of the rail.
[3] Rail head: the rail head is the top of the rail.
[4] Crosstie: rails are supported by cross pieces known as crossties (or ties or sleepers) that are generally perpendicular to the rails. These cross pieces are typically made of wood or concrete.
[5] Tie plate: the tie plate is a rectangular piece of steel on which the rail base sits. It rests on and is attached to the crosstie. The tie plate distributes the forces from the rail base to the area of the crosstie below the tie plate.
[6] Rail pad: when concrete crossties are used, the rail base rests on a rail pad, which rests on the top surface of the concrete crosstie. The rail pad provides a layer of protection between the rail base and the concrete crosstie surface. It is usually made of a polymer.
[7] Rail seat: in the context of concrete crossties, the rail seat is the area of the crosstie beneath the rail base.
[8] Fasteners: the tie plate is normally secured to the crosstie with fasteners, which can be spikes, or various screws, clips and clamps.
[9] Ballast: the crossties usually sit upon a bed of stones of a specific shape and size, called the ballast. The ballast supports the ties, plates/pads, rails and fasteners. It facilitates rainwater drainage, and provides a buffer against the encroachment of vegetation into the track area.
[10] Sunken tie plate: a tie plate is sunken when it has worn down the crosstie beneath it so that its lower surface is below the adjacent top surface of the crosstie. This is also referred to as plate cut.
[11] Misaligned tie plate: a misaligned tie plate usually occurs when a spike is loose or missing, thereby allowing the tie plate to rotate out of alignment with the rail.
[12] Rail seat abrasion: in the context of concrete crossties, rail seat abrasion occurs when rail vibration caused by trains wears down the rail pad and, once the rail pad is worn away, the top of the crosstie.
[13] Canadian Patent 2,572,082 [082 Patent], titled “System and Method for Inspecting Railroad Track”
, was issued to the Plaintiff and Defendant by Counterclaim, Georgetown Rail Equipment Company [Georgetown], on January 25, 2011. The patent application was open to public inspection as of January 12, 2006. The 082 Patent relates generally to a system and method for inspecting railroad track. It uses lasers, cameras and a processor to capture and analyze images of the railroad track in order to determine the distance between crossties, and detect misaligned or sunken tie plates.
[14] Canadian Patent 2,766,249 [249 Patent], titled “Tilt Correction System and Method for Rail Seat Abrasion”
, was issued to Georgetown on November 5, 2013. The patent application was open to public inspection as of December 29, 2010. The 249 Patent relates generally to a system and method for determining rail seat abrasion of a railroad track. It uses lasers, cameras and a processor to determine whether rail seat abrasion is present along the track.
[15] According to Georgetown, this case relates to systems and methods for determining the degree of wear of a wooden tie under a tie plate (the 082 Patent) and the degree of wear of a pad or a concrete crosstie under a rail (the 249 Patent). Both phenomena are hidden from view from above, because they occur underneath components that are visible from above. Both patents claim to solve this problem by comparing the height of the tie with the height of another track component: the tie plate and rail base, respectively. The 249 Patent also includes an algorithm for increasing the accuracy of the rail seat abrasion measurement by accounting for tilt.
[16] The Defendant and Plaintiff by Counterclaim, Tetra Tech EBA Inc [Tetra], has developed a system for inspecting railroad track which it calls the Three Dimensional Track Assessment System [3DTAS]. The 3DTAS is mounted on a rail car that moves along a railroad track. The system positions two lasers adjacent to the railroad track. It uses algorithms to analyze the features of a railroad track bed, including crossties, rails, rail bases, fasteners, ballast and spikes. These features are displayed on a three dimensional [3D] elevation map. Geographical location data may be identified using a Global Positioning System [GPS] receiver or an encoder. Tetra entered into an agreement to provide the 3DTAS and processing services to Canadian National Railway [CN], but ceased providing any services once this litigation was commenced.
[17] Tetra says that the 3DTAS infringes neither the 082 Patent nor the 249 Patent. Tetra has also challenged the validity of both Patents on the ground of obviousness.
[18] For the reasons that follow, I find that the 082 and 249 Patents are not invalid on the ground of obviousness. The identification of the particular problems, and the use of machine vision and specified calculations as possible solutions, required invention and were not obvious as of the relevant dates.
[19] The essential elements of the relevant 082 and 249 Patent claims are also present in the 3DTAS. Tetra’s sale of the 3DTAS to CN and its support of the system therefore infringed both Patents.
[20] Georgetown is a corporation organized and existing under the laws of the State of Texas, United States of America [US]. Georgetown provides track inspection services to numerous clients across North America, including most of the major railway companies.
[21] The Defendant Rail Radar Inc [Rail Radar] is a corporation organized and existing under the laws of the Province of Alberta. Rail Radar has not participated in this proceeding in any way, and its current status is unknown. Georgetown is not seeking relief against Rail Radar.
[22] Tetra is a corporation organized and existing under the laws of the Province of Alberta. Tetra provides services in various areas of transportation engineering, including infrastructure management and data collection for owners and operators of transportation infrastructure.
[23] Georgetown’s initial Statement of Claim was filed on May 29, 2015. Georgetown alleged infringement of approximately 55 claims of the 082 Patent and three claims of the 249 Patent, and sought an injunction, and damages or an accounting of profits.
[24] On July 15, 2015, Tetra filed its initial Statement of Defence denying liability. Georgetown filed its initial Reply on July 27, 2015.
[25] On May 26, 2016, Tetra filed its Amended Defence and Counterclaim, alleging that the 082 Patent and the 249 Patent are invalid due to obviousness.
[26] This proceeding was bifurcated by order of Prothonotary Kevin Aalto dated May 30, 2016. These reasons for judgment concern only the Liability Phase of the proceeding.
[27] On June 16, 2016, Georgetown filed its Amended Reply and Defence to Counterclaim, maintaining that the 082 Patent and the 249 Patent are valid and enforceable.
[28] On June 20, 2017, Georgetown filed a Fresh as Amended Statement of Claim, in which Georgetown no longer alleged infringement of the 082 Patent claims that relate to detecting the distance between crossties, or the breaks in a rail.
[29] Tetra filed its Re-Amended Statement of Defence on July 20, 2017.
[30] Georgetown filed its Further Amended Reply and Defence to Counterclaim on August 18, 2017.
[31] The 082 Patent describes the field of invention as follows:
The present invention relates generally to a system and method for inspecting railroad track and, more particularly to a system and method for inspecting aspects of a railroad track using a laser, camera, and a processor.
[32] The “Background of the Invention”
states that the majority of crossties in service are made of wood. Various other materials may be used, such as concrete, steel and composite or recycled materials, but these alternatives make up a relatively small percentage of all crossties. Over time, environmental factors may cause crossties to deteriorate until they must be replaced. Several million crossties are replaced in North America each year.
[33] The 082 Patent notes that railroad inspectors attempt to grade the condition of crossties and fastener systems on a regular basis. This grading is most often done with a visual inspection to identify crossties and fasteners that are rotten, broken, split or worn to an extent that their serviceable life is at its end. The process of visual inspection is time-consuming. In practice, inspection of the track is performed by an inspector walking along the track to inspect and record the conditions of the crossties and/or fasteners, which are spaced approximately every 20 inches along the track. According to one North American railway company, a crew of three or four inspectors can grade only between five and seven miles of track each day. The invention disclosed in the 082 Patent is intended to overcome, or at least reduce, this logistical challenge.
[34] The “Summary of the Disclosure”
of the 082 Patent describes a system and method for inspecting railroad track components:
The disclosed system includes lasers, cameras, and a processor. The lasers are positioned adjacent to the track. The laser emits a beam of light across the railroad track, and the camera captures images of the railroad track having the beam of light emitted thereon. The processor formats the images so that they can be analyzed to determine various measurable aspects of the railroad track. The disclosed system can include a GPS receiver or a distance device for determining location data. The measurable aspects that can be determined by the disclosed system include but are not limited to: the spacing between crossties, the angle of ties with respect to rail, cracks and defects in surface of ties, missing tie plates, misaligned tie plates, sunken tie plates, missing fasteners, damaged fasteners, misaligned fasteners, worn or damaged insulators, rail wear, gage or rail, ballast height relative to ties, size of ballast stones, and a break or separation in the rail. The system includes one or more algorithms for determining these measurable aspects of the railroad track.
[35] The 082 Patent then provides an explanation of different aspects of the invention, followed by a detailed description of specific embodiments and accompanying drawings. The claims of the 082 Patent, which number 80 in total, follow.
[36] The 249 Patent describes the field of invention as follows:
The present invention relates generally to systems and methods for inspecting railroad surfaces and, more particularly to systems and methods for determining rail seat abrasion via the utilization of tilt correction algorithms.
[37] The “Background of the Invention”
is initially similar to that provided in the 082 Patent. However, it goes on to explain that the construction of railroad tracks differs slightly depending on the type of tie material used. If wood ties are used, tie plates are placed on top of the ties, and rails are placed on top of the tie plates. If concrete ties are used, rails are placed on top of the ties with a thin polymer pad in between, preventing direct contact between the steel and the concrete.
[38] The 249 Patent states that normal railroad traffic causes friction between ties and rails, as well as rails and spikes, bolts, screws, or clips, and the surface under the ties. Of particular concern is friction at the point where the rail seat rests against the tie. Wear at this point, also known as rail seat abrasion, directly affects the life of the tie by causing it to loosen from the rail, despite the pads used between rails and concrete ties.
[39] According to the 249 Patent, railway companies monitor the wear of concrete ties either by direct manual measurement or through the use of electronic devices installed below individual railroad ties. However, this may be unreliable, hazardous, labour-intensive, complicated and disruptive to train traffic. The invention disclosed in the 249 Patent is intended to overcome, or at least reduce, these problems.
[40] The “Summary of the Disclosure”
of the 249 Patent describes a system and method for determining rail seat abrasion that uses lasers, cameras and processors in a manner similar to the system disclosed in the 082 Patent. However, the system is adapted to determine whether rail seat abrasion is present along the track. The processor employs a mathematics-based algorithm which compensates for tilt encountered as the inspection system moves along the track.
[41] The 249 Patent then provides a detailed description of specific embodiments and accompanying drawings. The claims of the 249 Patent, which number 18 in total, follow.
[42] Georgetown alleges infringement of claims 16 and 67; 37, 68 and 70; and 58, 71 and 73 of the 082 Patent. These claims pertain to a system and methods for detecting tie plate and crosstie contours:
[43] In greater detail, claim 16 describes:
A system for inspecting a railroad track bed, including the railroad track, to be mounted on a vehicle for movement along the railroad track, the system comprising:
at least one light generator positioned adjacent the railroad track for projecting a beam of light across the railroad track bed;
at least one optical receiver positioned adjacent the railroad track for receiving at least a portion of the light reflected from the railroad track bed and generating a plurality of images representative of the profile of at least a portion of the railroad track bed; and
at least one processor for analyzing the plurality of images and determining one or more physical characteristics of the said portion of the railroad track bed, the one or more physical characteristics comprising at least a geographic location of the plurality of images along the railroad track bed, wherein the processor includes an algorithm for detecting a misaligned or sunken tie plate of the railroad track bed, the algorithm comprising the steps of:
(a) analyzing a frame of the plurality of images, the frame comprising a region of interest;
(b) determining whether the region of interest contains a tie plate;
(c) if a tie plate is present, determining a crosstie contour and a tie plate contour;
(d) comparing an orientation of the crosstie contour and an orientation of the tie plate contour; and
(e) determining whether the tie plate is misaligned or sunken based upon the comparison.
[44] Claim 37 describes:
A method for inspecting railroad track bed, the railroad track bed including crossties, rails, associated fastening hardware, and ballast, the method comprising the steps of:
a) illuminating a line across the span of the railroad track bed;
b) receiving at least a portion of the light reflected from the railroad track bed;
c) generating a plurality of images representative of the profile of at least a portion of the railroad track bed;
d) analyzing the plurality of images and determining one or more physical characteristics of the said portion of the railroad track bed, the one or more physical characteristics comprising at least a geographic location of the plurality of images along the railroad track bed;
e) displaying the determined physical characteristics of the said portion of the railroad track bed; and
f) detecting a misaligned or sunken tie plate of the railroad track bed, the step of detecting comprising the steps of:
(a) analyzing a frame of the plurality of images, the frame comprising a region of interest;
(b) determining a whether the region of interest contains a tie plate;
(c) if a tie plate is present, determining a crosstie contour and a tie plate contour;
(d) comparing an orientation of the crosstie contour and an orientation of the tie plate contour; and
(e) determining whether the tie plate is misaligned or sunken based upon the comparison.
[45] Claim 58 describes:
A method of inspecting railroad track bed having a crosstie, rails, associated fastening hardware, and ballast, the method comprising the steps of:
a) traveling along the rails;
b) projecting a focused beam of light across the span of the railroad track bed;
c) capturing a plurality of images of the focused beam of light projected across a portion of railroad track bed while traveling along the rails;
d) determining one or more aspects of the portion of the railroad track bed by processing the plurality of images, the one or more aspects comprising at least a geographic location of the plurality of images along the railroad track bed;
e) outputting the determined one or more aspects of the portion of the railroad track bed; and
f) detecting a misaligned or sunken tie plate of the railroad track bed, the step of detecting comprising the steps of:
(a) analyzing a frame of the plurality of images, the frame comprising a region of interest;
(b) determining whether the region of interest contains a tie plate;
(c) if a tie plate is present, determining a crosstie contour and a tie plate contour;
(d) comparing an orientation of the crosstie contour and an orientation of the tie plate contour; and
(e) determining whether the tie plate is misaligned or sunken based upon the comparison.
[46] Claims 67, 68, 70, 71 and 73 are also in issue, but only insofar as they depend from the claims described above.
[47] Georgetown also alleges infringement of claims 7, 11 and 18 of the 249 Patent, which pertain to a system and method for detecting rail seat abrasion:
[48] In greater detail, claim 7 describes:
A method for determining rail seat abrasion of a rail road track, the method comprising the steps of:
(a) determining a height of a left rail base, right rail base, left crosstie and right crosstie, determining vertical pixel counts for each of the heights of the left rail base, right rail base, left crosstie and right crosstie and normalizing the vertical pixel counts based upon a measurement index;
(b) recording the heights of the left rail base, right rail base, left crosstie and right crosstie;
(c) determining an actual delta between the left rail base height and the left crosstie height and determining an actual delta between the right rail based height and the right crosstie height; and
(d) determining a rail seat abrasion value for the right and left rail bases.
[49] Claim 18 describes:
A system for determining rail seat abrasion of a rail road track, the system comprising:
at least one light generator positioned adjacent the rail road track, the light generator adapted to project a beam of light across the rail road track;
at least one camera positioned adjacent the rail road track for receiving at least a portion of the light reflected from the rail road track and for generating at least one image representative of a profile of at least a portion of the rail road track, and
at least one processor adapted to perform the steps comprising:
analyzing the at least one image;
determining a height of a left rail base, right rail base, left crosstie and right crosstie, determining vertical pixel counts for each of the heights of the left rail base, right rail base, left crosstie and right crosstie and normalizing the vertical pixel counts based upon a measurement index; and
determining whether rail seat abrasion is present along the rail road track.
[50] Claim 11 is also in issue, but only insofar as it depends from claim 7.
[51] There are two issues raised in the Liability Phase of this proceeding: whether the 082 and 249 Patents are valid; and, if so, whether the 3DTAS infringes the asserted claims of those Patents.
[52] Tetra alleges that the 082 and 249 Patents are invalid because the subject matter of the claims would have been obvious on the priority dates to a person skilled in the art, based on the common general knowledge existing one year before the priority dates. The parties agree that the priority dates, which are the same as the US provisional patent application filing dates, are June 30, 2004 for the 082 Patent, and June 23, 2009 for the 249 Patent. Georgetown maintains that the 082 and 249 Patents are both valid.
[53] Georgetown alleges that Tetra has infringed claims 16, 67, 37, 68, 70, 58, 71 and 73 of the 082 Patent, and claims 7, 11, and 18 of the 249 Patent. Tetra denies that the 3DTAS infringes the asserted claims of the Patents.
[54] Georgetown submitted the expert evidence of Dr. Harley Myler. Dr. Myler is a professor and Chair of the Electrical Engineering Department at Lamar University in Beaumont, Texas. He was qualified as an electrical engineer and expert in digital signal processing, in particular image processing, with a working knowledge of railways and track inspection techniques.
[55] Georgetown also called Gregory Thomas Grissom as a fact witness. Mr. Grissom has been the Chief Operating Officer of Georgetown for the past two years.
[56] Tetra submitted the expert evidence of Sébastien Parent. Mr. Parent is a physics engineer with over twenty years’ experience. He was qualified as a physics engineer and expert in machine vision integration, with first-hand experience in the field of image acquisition techniques and automated machine vision systems.
[57] Tetra also called Dr. Darel Edward Mesher as a fact witness. Dr. Mesher is an engineer and has been an employee of Tetra since approximately 1992. He was a driving force behind the development of the 3DTAS.
[58] Georgetown asks this Court to disregard or discount the evidence of Dr. Mesher on the ground that he is not impartial (citing Justice Frank Collier’s decision in Xerox of Canada Ltd v IBM Canada Ltd (1977), 33 CPR (2d) 24 at 38-40 (FCTD)). Tetra asks this Court to disregard or discount the evidence of Dr. Myler on similar grounds.
[59] I agree that both of these witnesses sometimes exhibited a tendency to provide answers, particularly in cross-examination, that were intended to bolster the position of the party that called them to testify, or undermine the position of the opposing party. This observation reflects more negatively on Dr. Myler than it does on Dr. Mesher. Dr. Myler was called as an expert witness, and therefore owed the Court a professional duty of impartiality. Dr. Mesher was called as a fact witness, and frankly acknowledged his interest in the success of the 3DTAS.
[60] Despite these reservations, I am not prepared to wholly reject or discount the evidence of either Dr. Mesher or Dr. Myler. Like other witnesses who were called to testify in this phase of the proceeding, they presented impressive qualifications and provided useful information. My reasons for preferring some witnesses’ evidence over that of others are explained in the analysis that follows.
[61] The first step in a patent suit is to construe the claims in order to give them meaning and determine their scope (Whirlpool Corp v Camco Inc, 2000 SCC 67 at para 43 [Whirlpool]). The relevant dates for construing the claims are the dates of publication of the patent applications: January 12, 2006 for the 082 Patent; and December 29, 2010 for the 249 Patent (Whirlpool at paras 54-55). The Court must examine the description contained in the patent to identify its “essential elements”
, and may be aided by expert evidence regarding the meaning of specific terms (Whirlpool at paras 43, 45, 57).
[62] The canons of claims construction may be found in the Supreme Court of Canada’s decisions in Whirlpool at paragraphs 49 to 55 and Free World Trust v Électro Santé Inc, 2000 SCC 66 [Free World Trust] at paragraphs 44 to 54. They are the following:
(a)claims are to be read in an informed and purposive way with a mind willing to understand, viewed through the eyes of the person skilled in the art as of the date of publication having regard to the common general knowledge;
(b)adherence to the language of the claims allows them to be read in the manner the inventor is presumed to have intended and in a way that is sympathetic to accomplishing the inventor’s purpose, which promotes both fairness and predictability; and
(c)the whole of the specification should be considered to ascertain the nature of the invention, and the construction of claims must be neither benevolent nor harsh, but should instead be reasonable and fair to both the patentee and the public.
[63] In order to construe the claims in issue, the Court must define the Person of Ordinary Skill in the Art [POSITA]. This is “the person to whom the patent is said to be addressed, through whose eyes the Court is to read the patent, and who stands as the criterion for determination of obviousness”
(Amgen Canada Inc v Apotex Inc, 2015 FC 1261 at para 42).
[64] Georgetown describes the POSITA for the 082 and 249 Patents as an electrical or computer engineer who has at least three years of experience working with image processing systems, or a Master’s degree, and with a working knowledge of railways and track inspection techniques.
[65] Tetra maintains that the POSITA for the 082 Patent is a person with a degree in engineering or physics with five to seven years of experience in the field of machine vision. For the 249 Patent, Tetra says that the POSITA is again a person with a degree in engineering or physics, but with less practical experience given the more restricted application of the 249 Patent, and because more became known about machine vision between the publication dates of the 082 and 249 Patents.
[66] The critical difference between the parties’ positions is the degree to which the POSITA must possess a working knowledge of railways and track inspection techniques.
[67] I prefer the articulation of the POSITA advanced on behalf of Tetra. Every claim of the 082 and 249 Patents is premised on the use of machine vision. It follows that the POSITA must understand the use of machine vision to inspect surfaces. The Patent refers to “tool boxes”
and “known software packages”
, both of which potentially encompass machine vision and image processing beyond the context of railways. Indeed, the 082 Patent acknowledges that the techniques may be applied in other contexts. A knowledge of railways is therefore ancillary to a knowledge of the manner in which machine vision techniques may be applied in different contexts.
[68] The patent must be construed taking into account the “common general knowledge”
shared by persons skilled in the art (Free World Trust at para 44; Whirlpool at para 53). This is the knowledge possessed by the POSITA at the relevant time, and includes what the POSITA would reasonably be expected to know (Sanofi-Synthelabo Canada Inc v Apotex Inc, 2008 SCC 61 at para 70 [Sanofi-Synthelabo]; Whirlpool at para 74). The common general knowledge of the POSITA must be established on a balance of probabilities and cannot be assumed (Uponor AB v Heatlink Group Inc, 2016 FC 320 at para 47.
[69] The assessment of the common general knowledge is governed by the principles found in Eli Lilly & Co v Apotex Inc, 2009 FC 991 at paragraph 97 and General Tire & Rubber Co v Firestone Tyre & Rubber Co, [1972] RPC 457 (UKHL) at pages 482 to 483:
(a)the common general knowledge imputed to the POSITA must be carefully distinguished from what in patent law is regarded as public knowledge;
(b)common general knowledge is a different concept derived from a common sense approach to the practical question of what would in fact be known to an appropriately skilled addressee – the sort of person, good at his or her job, who could be found in real life;
(c)individual patent specifications and their contents do not normally form part of the relevant common general knowledge, although there may be specifications which are so well known that they do form part of the common general knowledge, particularly in certain industries;
(d)regarding scientific papers generally:
i.it is not sufficient to prove common general knowledge that a particular disclosure is made in an article, or series of articles, or in a scientific journal, no matter how wide the circulation of that journal may be, in the absence of any evidence that the disclosure is accepted generally by those who are engaged in the art to which the disclosure relates;
ii.a piece of particular knowledge as disclosed in a scientific paper does not become common general knowledge merely because it is widely read, and still less because it is widely circulated;
iii.such a piece of knowledge only becomes general knowledge when it is generally known and accepted without question by the bulk of those who are engaged in the particular art; in other words, when it becomes part of their common stock of knowledge relating to the art; and
iv.it is difficult to appreciate how the use of something which has in fact never been used in a particular art can ever be held to be common general knowledge in the art.
[70] Georgetown objects to the Court’s consideration of Appendix SP-09 to Mr. Parent’s expert report (Daniel L Magnus, “Non-contact technology for track speed rail measurement: ORIAN”
(Paper delivered at the Nondestructive Evaluation of Aging Railroads, 30 June 1995), 2458 SPIE 45 [Appendix SP-09]) as prior art in the obviousness analysis for the 082 and 249 Patents, and consideration of the 082 Patent as prior art in the obviousness analysis for the 249 Patent. Georgetown asserts that these documents were not specifically pled in Tetra’s Counterclaim.
[71] In response, Tetra argues that these documents were included in Mr. Parent’s expert report and were responded to by Georgetown. They therefore form a part of the record, and the Court has a wide discretion to evaluate evidence on the record. Alternatively, in the course of closing submissions, counsel for Tetra offered to amend the pleading.
[72] When a party pleads invalidity, as Tetra has done in its Statement of Defence and Counterclaim, especially where the invention is complex, it is generally accepted that the party must identify in its pleading the prior art that supports the allegation of obviousness (Throttle Control Tech Inc v Precision Drilling Corp, 2010 FC 1085 at para 13). It is possible to remedy a defect in a pleading by amendment, and it may be an error for a judge to refuse a reasonable request to amend (Janssen Inc v Abbvie Corp, 2014 FCA 242 [Janssen]).
[73] A technical approach to the question is to be avoided. In the words of the Federal Court of Appeal in Janssen (at para 3, citing Continental Bank Leasing Corp v R, [1993] TCJ No 18):
[…] I prefer to put the matter on a broader basis: whether it is more consonant with the interests of justice that the withdrawal or amendment be permitted or that it be denied. The tests mentioned in cases in other courts are of course helpful but other factors should also be emphasized, including the timeliness of the motion to amend or withdraw, the extent to which the proposed amendments would delay the expeditious trial of the matter, the extent to which a position taken originally by one party has led another party to follow a course of action in the litigation which it would be difficult or impossible to alter and whether the amendments sought will facilitate the court’s consideration of the true substance of the dispute on its merits. No single factor predominates nor is its presence or absence necessarily determinative. All must be assigned their proper weight in the context of the particular case. Ultimately, it boils down to a consideration of simple fairness, common sense and the interest that the courts have that justice be done. [Emphasis original]
[74] In this case, the prior art relied on by Tetra was disclosed in the expert report of Mr. Parent, which was delivered to Georgetown approximately four months before the commencement of trial. Georgetown was aware of all of the prior art on which Tetra intended to rely, including that which was not pleaded, and chose to respond through the expert report of Dr. Myler. Georgetown has not demonstrated any prejudice resulting from Tetra’s failure to include these documents in its Counterclaim. Furthermore, as will be seen below, the contested prior art is not central to the Court’s obviousness analysis.
[75] I therefore exercise my discretion to permit Tetra to rely on all of the prior art cited in the expert report of Mr. Parent.
[76] According to Mr. Parent, a great deal became known about the field of machine vision in the 1990s. This included the use of optical 3D sensors and 3D laser triangulation. A common technique was to use machine vision to determine the appearance of an object under normal circumstances, and then detect and measure any anomalies or other features of interest. The visual characteristics of an object or scene were understood to be critical to the design of an image acquisition system. Those characteristics included, but were not limited to, colour, type of reflectivity (diffuse or specular), size of the scene, smallest detail of interest and speed of motion. This methodology had been well established since the early 1990s (see, for example, Kevin Harding, “The Art of Lighting Science” Vision Online (28 April 2000) [Appendix SP-06]).
[77] Mr. Parent explained that, at the relevant times, many techniques existed to acquire two dimensional [2D] or 3D information from an object or scene. He observed that a 3D triangulation technique was a good and natural choice for the tasks described in the 082 and 249 Patents. The 082 Patent refers to the use of specialized 3D triangulation cameras. Mr. Parent described this as a well-known technique for 3D measurement at the time the applications for both Patents were filed.
[78] One of the prior publications cited by Mr. Parent, Liviu Bursanescu & François Blais, “Automated Pavement Distress Data Collection and Analysis: a 3-D Approach”
(1997) 41574 NRC 311 [Appendix SP-07], describes a system which uses triangulation with infrared lasers and cameras; uses a beam of laser light with an angular expanse; is mounted on a vehicle; adapts the acquisition configuration (geometry) and number of devices to the need (i.e., pavement inspection); includes an optical encoder and a GPS for geographic coordinates; includes an inclinometer for road gradient and crossfall (i.e., road camber); corrects the profile for roll and pitch of the vehicle; includes a real-time processor for feature detection; includes a storing device; includes a post-processing device extracting and classifying features; and is used to inspect road surfaces and to identify defects.
[79] Mr. Parent acknowledged that Appendix SP-07 does not relate to railways. However, he maintained that machine vision techniques need not be linked to a particular field of application. Indeed, paragraph 0024 of the 082 Patent refers to the possible application of the disclosed invention to road, electrical line, piping or other network inspection.
[80] Mr. Parent also cited Denis Gingras, “Optics and Photonics Used in Road Transportation”
(Paper delivered at the Opto-Contact: Workshop on Technology Transfers, Start-Up Opportunities and Strategic Alliances, 24 September 1998), 3414 SPIE 264 [Appendix SP-08]. The article describes a road inspection system that uses a laser-based triangulation system mounted on a vehicle, with an odometer and GPS for localization of the scans. Mr. Parent noted that the processing includes calibration correction due to the tilt and roll of the vehicle, which he compared to the technique employed by the 249 Patent.
[81] Mr. Parent identified a number of articles and patents directed to the context of railways, including Appendix SP-09, which concerns an optical rail profile measurement technology. The system analyzes rail wear, is mounted underneath a track inspection vehicle, uses a combination of charged coupled device cameras and laser diodes to acquire video images, employs a structured light source, performs analysis of the left and right rail profile, analyzes the images using a computer, translates each rail image into real-world X-Y coordinates, and uses an encoder to synchronize each rail measurement with a known location.
[82] Other examples of prior art adduced by Mr. Parent include:
(a)George Kantor et al, “Automatic Railway Classification using Surface and Subsurface Measurements”
(Paper delivered at the Proceeding of the 3rd International Conference on Field and Service Robotics, January 2001), Robotics Institute of Carnegie Mellon University [Appendix SP-10], which describes methods for evaluating railroad health by analyzing features visible from the surface of the railway, and analyzing subsurface measurements to assess ballast health using ground-penetrating radar;
(b)“Device for Identifying Track Structures”
, Japanese Patent No H06-322707 (13 May 1993) [Appendix SP-11], which describes a device that identifies rail track structures, including rail ties (sleepers) and ballast. The device is mounted on a rail car and uses a slit light source to illuminate the track, and a camera to capture the portion of the track illuminated by the light source. The resulting image is stored and processed in order to identify the position of the sleeper, ballast and/or rail joint.
(c)“Device and Method for Detecting Slippage of Rail Clamping Device and Method for Detecting Position of Rail”
, Japanese Patent No 11-172606 (9 December 1997) [Appendix SP-12], which describes a system to detect “slippage”
of rail clamping devices, including spikes and bolts. The system is affixed to the bottom of a vehicle that runs on rails, and includes sensors for outputting signals corresponding to the shape of the rail clamping devices; and
(d)G van der Merwe, “IM2000 Infrastructure Measuring Car: the application of recording results”
(2001) 30-4 Rail Engineering International 14 [Appendix SP-13], which describes a system mounted on a rail car that is capable of measuring, processing, and storing data concerning the condition of track geometry, catenary and rails. The system is capable of taking measurements at speeds of up to 120 km/h. A laser creates a plane of light surrounding the rail. Using a high-resolution camera, the image of the full rail profile is acquired, and is then converted and analyzed by a computer to calculate the extent of rail wear.
[83] Georgetown concedes that the POSITA would be familiar with machine vision lighting techniques and general techniques used to scan surfaces using cameras and a light source, as described in Appendices SP-07 and SP-08. However, Georgetown disagrees that the POSITA would be familiar with the specific applications of this technology, particularly regarding the inspection of pavement, roads and railway tracks, as described in Appendices SP-09 to SP-13.
[84] Dr. Myler did not identify any publications as illustrative of the common general knowledge of the POSITA at the time of publication of the 082 and 249 Patents. Instead, he spoke in generalities:
As railroad technology improved, trains became larger, faster and capable of handling heavier loads. The mechanical integrity of the track, although always a concern, became even more acute given the potential for railroad disasters such as derailments. Human inspectors who were specifically trained to spot problem areas in need of attention performed the inspection of railroad track. At first, these inspectors would walk along the track and later, they would use special purpose vehicles to perform their inspections …
[85] In the portion of his expert report titled “Background and common general knowledge of the skilled person”
, Dr. Myler listed only the physical characteristics of the railroad track and track bed that are of interest to rail transport entities, and the commercial and governmental organizations that maintain and use them.
[86] I have no hesitation in adopting the approach advocated by Tetra. I see no reason to limit the assessment of the common general knowledge of the POSITA to prior art existing within the limited context of railways. The primary focus of both the 082 Patent and the 249 Patent is the use of machine vision to address well-known challenges associated with the inspection of railway tracks. The POSITA would therefore look to the application of machine vision to the inspection of railways, as well as other comparable surfaces such as roads and pavement.
[87] Tetra must establish that the prior art on which it relies was publicly available through a reasonably diligent search as of the relevant date (E Mishan & Sons Inc v Supertek Canada Inc, 2015 FCA 163 at para 20, citing Apotex Inc v Sanofi-Aventis, 2011 FC 1486). Georgetown disputes that the prior art cited by Mr. Parent could have been located through a reasonably diligent search.
[88] I am persuaded that the prior art on which Tetra relies would have been located through a reasonably diligent search by the POSITA. Appendices SP-08 and SP-09 were published by SPIE, an organization that is well-known in the field of machine vision, and which has previously published Dr. Myler’s own work. Appendix SP-09 elaborates on Appendix SP-13. Appendix SP-10 was published by the Robotics Institute of Carnegie Mellon University, which Dr. Myler described as a “wellspring” of information. The technology described in Appendix SP-11 does not differ markedly from that found in the other prior art cited by Tetra.
[89] Individual patent specifications and their contents do not normally form part of the relevant common general knowledge. Here, we are dealing with a specific application of a known technology to a specific industry, i.e., railways. Nevertheless, I have some doubt whether the patent specifications cited by Mr. Parent were sufficiently well-known to form a part of the common general knowledge at the relevant times. I note, however, that Tetra relies less on the particulars of the patents cited by Mr. Parent, and more on the general principle that machine vision techniques may be applied to the inspection of railways. This potential application of machine vision would clearly form a part of the common general knowledge.
[90] I therefore conclude that, at a minimum, the POSITA would have understood the following as common general knowledge at the relevant dates for claim construction:
(a)optical 3D sensors and 3D laser triangulation could be used to determine the appearance of an object under normal circumstances, and then detect and measure any anomalies or other features of interest;
(b)surfaces could be inspected and defects could be identified using a system with the following attributes:
i.triangulation with infrared lasers and cameras;
ii.a beam of laser light with an angular expanse;
iii.mounted on a vehicle;
iv.adapting the acquisition configuration (geometry) and number of devices to the need;
v.including an optical encoder and a GPS for geographic coordinates;
vi.including an inclinometer for gradient and camber;
vii.correcting the profile for roll and pitch of the vehicle;
viii.including a real-time processor for feature detection;
ix.including a storing device; and
x.including a post-processing device to extract and classify features; and
[91] Patent construction is a matter of law for the judge. Expert evidence is necessary only where the meaning of a term is not apparent based on a reading of the patent specification (Johnson & Johnson Inc v Boston Scientific Ltd, 2008 FC 552 at para 92). I have found expert evidence to be helpful in construing the following disputed terms.
[92] Georgetown says that the term “frame” is “derived from a picture frame and is typically rectangular. A frame is a single image or a part of a single image. In image processing, a frame is a collection or data structure of pixels”
.
[93] Tetra maintains that the term “frame” as used in the 082 and 249 Patents refers specifically to a 2D image.
[94] While the distinction between a 2D image and a 3D image is central to the dispute between the parties regarding infringement, they appear to have a common understanding of the meaning of “frame”. A frame is a single image or a part of a single image. Viewed in isolation, a frame is a collection or data structure of pixels that may be displayed as a 2D image.
[95] Georgetown states that “analyzing a frame of the plurality of images” encompasses any examination or evaluation of the frame. According to Georgetown, this is a form of image analysis, a sub-discipline of signal processing, which involves the extraction of meaningful information from images. In this context, analyzing “a” frame includes analyzing “many” frames.
[96] Tetra says that “analyzing a frame of the plurality of images” constitutes a frame-by-frame analysis of 2D images.
[97] At this stage of the analysis, I do not see a significant difference in the positions taken by the parties. I am satisfied that “analyzing a frame of the plurality of images” entails examining or evaluating one or many “frames”.
[98] Georgetown defines a “region of interest” [ROI] as a bounded set of pixels that define an area being processed. An ROI could be an entire image frame or any bounded set of pixels therein. Usually, ROIs are used to restrict the focus from a large area to a smaller area, in order to reduce processing effort and improve efficiency or speed. In other words, the processing is restricted to a certain area, because that area likely contains the information being sought.
[99] Tetra similarly defines an ROI as an area to which a view is restricted, in order to find something of interest and improve processing. However, Tetra stipulates that there is no processing done to find that region.
[100] To the extent that there is any material difference in the positions taken by the parties, I prefer the approach advocated by Georgetown. An ROI is a bounded set of pixels that defines an area to be processed. Some degree of processing may be required to identify the region in question.
[101] Georgetown defines “contour” as an outline, an edge, a line or a surface that represents the profile of an object.
[102] Tetra agrees with this definition, but stipulates that contour, as the term is used in the 082 and 249 Patents, can only be determined on a 2D image.
[103] Georgetown says that Items 10 and 14 in Figures 7A and 7B in the 082 Patent are examples of crosstie contours and tie plate contours, respectively. They represent the contour of the tie and the edge of the tie plate viewed in cross-section. In my view, this is a fair articulation of the manner in which “contour” is used in the 082 and 249 Patents.
[104] The term “actual delta” appears only in the 249 Patent. Georgetown defines “actual delta” as the distance or difference between two points. Georgetown says that the “actual delta” is derived using formulas that include a tilt correction factor [TC], as disclosed in the 249 Patent:
TC = (HLrail – HRrail)(.12)
ΔleftRail = (HLrail – HLtie) – TC
ΔrightRail = (HRrail – HRtie) + TC
[105] According to Georgetown, when the rails are level, TC equals 0. TC is therefore not a necessary component of determining the “actual delta”.
[106] Tetra argues that the invention disclosed by the 249 Patent is fundamentally concerned with TC, and the algorithm for determining the actual delta necessarily requires a consideration of TC.
[107] According to the “Summary of Invention”
contained in the 249 Patent, the invention comprises “an inspection system comprising lasers, cameras, and processors adapted to determine whether rail seat abrasion is present along the track. The processor employs a mathematics based algorithm which compensates for tilt encountered as the inspection system moves along the track”
. I agree with Georgetown that a plain reading of the 249 Patent does not contemplate that compensation for tilt will always be necessary as the system moves along the track. Instead, the system compensates for tilt only when this phenomenon is “encountered”
.
[108] It follows that the algorithm for determining the actual delta does not necessarily require a consideration of TC in all circumstances where the system is used. I therefore prefer the construction of “actual delta” advocated by Georgetown.
[109] Subsection 43(2) of the Patent Act states that a patent is presumed to be valid in the absence of evidence to the contrary. A party alleging invalidity bears the burden of establishing this on a balance of probabilities. The burden therefore falls upon Tetra.
[110] Pursuant to s 28.3 of the Patent Act, a patent cannot be issued for an invention that was obvious on the priority date to a person skilled in the art or science to which the patent pertains. The parties agree that obviousness is to be assessed as of June 30, 2004 for the 082 Patent, and June 23, 2009 for the 249 Patent.
[111] Obviousness is generally considered to be a factual determination, or a question of mixed fact and law (Wenzel Downhole Tools Ltd v National-Oilwell Canada Ltd, 2012 FCA 333 at para 44 [Wenzel]). It must be assessed on a claim-by-claim basis (Zero Spill Systems (Int’l) Inc v Heide, 2015 FCA 115 at para 85).
[112] When considering obviousness, hindsight is prohibited. To determine whether a claim is obvious, courts generally follow the four-part test found in Sanofi-Synthelabo at paragraph 67:
(a)identify the notional “person skilled in the art”
and the relevant common general knowledge of that person;
(b)identify the inventive concept of the claim in question or, if that cannot readily be done, construe it;
(c)identify what, if any, differences exist between the matter cited as forming part of the “state of the art”
and the inventive concept of the claim or the claim as construed; and
(d)viewed without any knowledge of the alleged invention as claimed, do those differences constitute steps which would have been obvious to the person skilled in the art or do they require any degree of invention?
[113] The fourth step of the inquiry may require consideration of whether the claimed invention was “obvious to try”
based on the following non-exhaustive factors (Sanofi-Synthelabo at para 69):
(a)is it more or less self-evident that what is being tried ought to work? Are there a finite number of identified predictable solutions known to persons skilled in the art?
(b)what is the extent, nature and amount of effort required to achieve the invention? Are routine trials carried out or is the experimentation prolonged and arduous, such that the trials would not be considered routine?
(c)is there a motive provided in the prior art to find the solution the patent addresses?
[114] While the “obvious to try”
test may be invoked in a mechanical field, its consideration is not always required (Wenzel at para 95). The analysis tends to arise in areas of endeavour where advances are made through experimentation, and where numerous interrelated variables may affect the desired result, e.g., the development of pharmaceuticals (Sanofi-Synthelabo at para 68).
[115] Tetra did not place significant emphasis on the “obvious to try”
test. In closing submissions, counsel for Tetra conceded that the analysis may not be necessary in this case. I agree. The evidence does not establish that machine vision and railway inspection are areas of endeavour where advances are typically made through experimentation, and where numerous interrelated variables may affect the desired result.
B. Developments Leading to the Patents
[116] Georgetown was incorporated in 1993. Its sole line of business is the provision of technology, products and services to railroads. Georgetown commenced research into track inspection technologies in 2003, and created a new business unit for this purpose. The initial concept was laser-profiling technology mounted on a vehicle. The product was almost immediately commercialized. Georgetown’s goal was to convert the rail industry to automated tie inspection. At the time, railroads were typically inspected by individuals who walked along the track. Walking inspections were so ingrained in the industry that Georgetown’s idea of automating inspection encountered significant resistance, which continued as late as 2011.
[117] The 082 Patent application entered the Canadian National Phase on December 28, 2006, based on Patent Application No. PCT/US2005/023132, which had been filed on June 30, 2005, and claimed priority from US provisional patent application No. 60/584,769 filed on June 30, 2004. The 082 Patent application was opened to public inspection on January 12, 2006. According to Georgetown, the problem of rail seat abrasion first came to light around 2006, after several derailments in the Pacific Northwest. This resulted in the development of the technology described in the 249 Patent. The 249 Patent application entered the Canadian National Phase on December 21, 2011, based on Patent Application No. PCT/US2010/025004, which had been filed on February 23, 2010, and claimed priority from US provisional patent application No. 12/489,570 filed on June 23, 2009. The 249 Patent application was opened to public inspection on December 29, 2010.
[118] Today, Georgetown has a fleet of 15 cars equipped with its patented technology. Georgetown offers track inspection services across North America for various national, regional, short-line and commuter railroads. It has no competitors in the US. Tetra is its only competitor in Canada.
[119] The POSITA and the common general knowledge are discussed under the heading Claims Construction, above.
[120] The Federal Court of Appeal has recently observed that there may be cases in which the inventive concept may be grasped without difficulty; however, because “inventive concept”
is undefined, the search for it has brought considerable confusion into the law of obviousness. That uncertainty may be reduced by avoiding the inventive concept altogether, and pursuing the alternative course of construing the claim. This avoids distraction or engaging in an unnecessary “satellite debate”
(Ciba Specialty Chemicals Water Treatments Limited v SNF Inc, 2017 FCA 225 at para 77).
[121] According to Tetra, the claims of the 082 Patent may be separated into three groups, with three independent claims for each group:
(a)Group 1 – claims 1, 22, 43: these relate to methods and a system to determine the distance between crossties;
(b)Group 2 – claims 16, 37 and 58: these relate to methods and a system to detect misaligned or sunken tie plates; and
(c)Group 3 – claims 64, 65 and 66: these relate to methods and a system to identify a break in a rail.
[122] Georgetown proceeded to trial only in respect of the claims in Group 2: those that relate to methods and a system to detect misaligned or sunken tie plates. However, Tetra is seeking a declaration of invalidity of all claims of the 082 Patent, as well as claims 7, 11 and 18 of the 249 Patent. The Court must therefore identify the inventive concepts of, or alternatively construe, all of the claims challenged by Tetra, not just those that are alleged to be infringed by the 3DTAS.
[123] According to Georgetown, the inventive concepts of Group 2 – claims 16, 37 and 58 of the 082 Patent (misaligned or sunken tie plates) – are a system and methods that use a laser to shine a light on the railway track bed and a camera to capture images of the profile of the track bed. The system and methods include a processor with an algorithm for analyzing the images in a particular manner to detect a misaligned or sunken tie plate. The algorithm (a) analyzes a frame of the many images having a region of interest; (b) determines the presence of a tie plate in the region of interest; (c) determines the crosstie contour and tie plate contours; (d) compares the orientation of the crosstie contour and orientation of the tie plate contour; and (e) determines whether the tie plate is misaligned or sunken based upon the comparison.
[124] With respect to Group 1 – claims 1, 22 and 43 (distance between crossties) – the 082 Patent acknowledges that determining whether a frame has a crosstie or not can be performed by imaging techniques known in the art. However, Georgetown maintains that a system or methods for measuring the distance between ties by counting the number of frames and using the speed of the vehicle to calculate the distance is inventive.
[125] With respect to Group 3 – claims 64, 65 and 66 (break in a rail) – Georgetown asserts that a system or methods for measuring the gap between adjoining rails by counting the number of images taken between the ends of the rails and using the speed of the car to calculate the gap distance is inventive.
Georgetown says that the inventive concept of claim 7 of the 249 Patent is a method for determining rail seat abrasion of a railroad track using the steps of (a) determining the heights of the left and right rail bases, and left and right crossties; (b) determining vertical pixel counts for those heights and normalizing the vertical pixel counts upon a measurement index; (c) recording those heights; (d) determining the actual delta between the left rail base height and left crosstie height, and between the right rail base height and right crosstie height; and (e) determining rail seat abrasion values for the left and right rail bases.
[127] According to Georgetown, the inventive concept of claim 11 is the same as that of claim 7, but with step (e) accomplished by using the actual delta for the left and right rail bases. The inventive concept of claim 18 is a system for determining rail seat abrasion having (a) light generators and cameras to generate images representative of the profile of the railroad track bed; and (b) a processor that (i) analyzes the images, (ii) determines the heights of the left and right rail bases, and left and right crossties, (iii) determines vertical pixel counts for those heights and normalizes the vertical pixel counts upon a measurement index, and (iv) determines whether rail seat abrasion is present along the railroad track using the measurements.
[128] Tetra’s expert witness, Mr. Parent, testified that the standard features of all automated railway inspection systems are (a) a light generator projecting a beam spanning across the railroad; (b) an optical receiver generating images representative of the profile of the railway track; (c) a processor using algorithms to analyze the images; and (d) a capacity to link an image captured by the receiver to a geographical location. He expressed the view that the image acquisition system described in both the 082 and 249 Patents is a standard 3D triangulation technique by line projection, an approach that was well-known in 2004. Mr. Parent evaluated the 082 Patent as distinct at the algorithm level only, but declined to characterize this as an inventive concept. He denied the existence of any inventive concept of the 249 Patent.
[129] In my view, Tetra’s assertion that the 082 and 249 Patents are inventive, if at all, only in respect of their algorithms is broadly consistent with Georgetown’s characterization of the inventive concepts. I would therefore define the inventive concepts as follows:
•082 Patent, Group 1 – claims 1, 22, 43 (distance between crossties): a machine vision system that measures the distance between ties by counting the number of frames and using the speed of the vehicle to calculate the distance.
•082 Patent, Group 2 – claims 16, 37 and 58 (misaligned or sunken tie plates): a machine vision system that (a) analyzes a frame of the many images having a region of interest; (b) determines the presence of a tie plate in the region of interest; (c) determines the crosstie contour and tie plate contours; (d) compares the orientation of the crosstie contour and orientation of the tie plate contour; and (e) determines whether the tie plate is misaligned or sunken based upon the comparison.
•082 Patent, Group 3 – claims 64, 65 and 66 (break in a rail): a machine vision system that measures the gap between adjoining rails by counting the number of images taken between the ends of the rails and using the speed of the car to calculate the gap distance.
•249 Patent (rail seat abrasion): a machine vision system that (a) analyzes images; (b) determines the heights of the left and right rail bases, and left and right crossties, applying TC as needed; and (c) determines the presence of rail seat abrasion using those measurements.
[130] Georgetown concedes that the prior art cited by Tetra demonstrates that machine vision systems were available at the relevant times to capture images of railway track bed components, and to measure their relative positions. However, Georgetown says that none of the prior art would have led a POSITA to build a laser vision system to detect sunken tie plates or rail seat abrasion, or to use the precise series of steps or calculations claimed in the 082 and 249 Patents. Georgetown argues that none of the prior art cited by Tetra discusses TC, or makes use of an actual delta between two points to identify rail seat abrasion.
[131] The following charts prepared by Dr. Myler summarize the elements of the 082 Patent that are said to be missing from the prior art cited by Tetra:
[132] The following chart prepared by Dr. Myler summarizes the elements of the 249 Patent that are said to be missing from the prior art cited by Tetra:
[133] Georgetown devoted most of its evidence and argument to defending the validity of the claims falling within Group 2 (misaligned or sunken tie plates). With respect to Group 1 (distance between crossties), Dr. Myler said only that none of the prior art cited by Tetra identified any problem relating to measuring the distance between ties, or suggested any solution such as that found in claims 1, 22 and 43 of the 082 Patent.
[134] Similarly, with respect to Group 3 (break in a rail), Dr. Myler said only that none of the prior art cited by Tetra described a system or method for measuring the gap between adjoining rails by counting the number of images taken between the ends of the rails, and using the speed of the car to calculate the gap distance. He expressed the view that none of the prior art documents identified any problem relating to measuring the gap distance, or suggested any solution such as that found in claims 64, 65 and 66 of the 082 Patent.
[135] Tetra does not take serious issue with Georgetown’s assessment of the differences between the prior art and the inventions claimed by the 082 and 249 Patents. Instead, Tetra describes Georgetown’s efforts to distinguish its patented inventions from the prior art as “strategic”
and “unfair”
. Tetra notes that Georgetown’s original Statement of Claim alleged infringement of 55 claims of the 082 Patent, and claims 7, 11 and 18 of the 249 Patent. Georgetown amended its Statement of Claim on June 20, 2017, shortly before the parties were due to exchange their initial expert reports. The effect of the amendments was to discontinue the action in relation to the claims falling within Group 1 (distance between crossties) and Group 3 (break in a rail).
[136] I accept Georgetown’s characterization of the differences between the prior art and the claimed inventions which, in my view, is consistent with the evidence adduced in this proceeding. The disagreement between the parties is not so much whether the claimed inventions differed from the prior art, but whether the differences were obvious.
[137] It is common ground between the parties that machine vision and 3D triangulation techniques, assisted by software, were available and commonly used to examine height differences and other elements of various surfaces. This process was applied in many different contexts. Tetra notes that the first section of all claims in both the 082 and 249 Patents is the same regardless of the surface in issue. Only the processing algorithm changes.
[138] Tetra says that it would have been obvious for the POSITA to use 3D triangulation to inspect different features of the railway bed. The industry would inevitably adapt known automated surface examination techniques to the inspection of rail components. Prior art demonstrated that machine vision, specifically 3D triangulation techniques, could be applied in the context of railways. However, Tetra acknowledges that none of the prior art was concerned with assessing plate cut or rail seat abrasion.
[139] Tetra nevertheless argues that, prior to 2004, the POSITA would have reason to combine common techniques known in the fields of machine vision and 3D triangulation to arrive at the claimed inventions. There was a strong commercial incentive to automate rail inspection, as illustrated by the collaboration between Dr. Mesher and CN that ultimately resulted in the 3DTAS.
[140] With respect to the 249 Patent, Tetra says that there are no inventive elements or steps that were not already disclosed by the 082 Patent, with the possible exception of TC. The content of the 249 Patent is identical to that of the 082 Patent, as evidenced by Figures 1 to 12 and the corresponding descriptive paragraphs. According to Tetra, if claims 7, 11 and 18 of the 249 Patent do not account for tilt, then they lack novelty given the teachings of the 082 Patent. The 082 Patent gives the POSITA all the information necessary to measure the heights of the top of the rail and the base of rail, and to calculate the difference.
[141] Georgetown responds that there are significant differences between the 082 Patent and the common general knowledge and prior art with respect to determining plate cut. None of the prior art mentions plate cut as a problem to be solved; nor does it provide a means of determining plate cut using a machine vision system.
[142] With respect to the 249 Patent, Georgetown asserts that none of the prior art documents recognize a problem relating to rail seat abrasion; nor do they suggest any solution to determining the presence of rail seat abrasion or its extent. Georgetown therefore maintains that there is nothing in the prior art that would have caused the POSITA to evaluate rail seat abrasion using the system or methods described in the 249 Patent. Rail seat abrasion primarily afflicts concrete ties, which are not the focus of the prior art.
[143] Georgetown notes that another element of claims 7 and 11 of the 249 Patent is the determination of an actual delta. To the extent that Tetra argues this must be understood as encompassing TC, none of the prior art discusses TC or otherwise determines an actual delta between two points to measure rail seat abrasion.
[144] With respect to the remaining claims of the 082 Patent, i.e., those pertaining to Groups 1 and 3, Georgetown argues that none of the prior art recognizes any problem relating to measuring the distance between ties or the gaps between adjoining rails. Nor does it suggest solutions similar to those found in the 082 Patent.
[145] The onus is on Tetra to establish, on a balance of probabilities, that the inventions claimed by the 082 and 249 Patents were obvious as of June 30, 2004 and June 23, 2009, respectively. I am not persuaded that Tetra has met this burden.
[146] It is true that, prior to 2004, machine vision and 3D triangulation techniques, assisted by software, were available and commonly used to examine height differences and other elements of various surfaces. The prior art cited by Tetra includes several documents that applied this technology in the context of railways. However, none of the prior art identifies plate cut or rail seat abrasion as problems to be solved, or suggests solutions similar to those disclosed by the 082 and 249 Patents.
[147] As noted by Georgetown, plate cut and rail seat abrasion are phenomena that are hidden when viewed from above, because they occur underneath components that are visible from above. Both patents solve this problem by comparing the height of the tie with the height of another track component: the tie plate and rail base, respectively. The 249 Patent also includes an algorithm for increasing the accuracy of the rail seat abrasion measurement by accounting for tilt. Neither the existence of these problems nor the patents’ proposed solutions are evident in the prior art. Nor could they have been arrived at without inventive insight.
[148] Tetra’s allegation of invalidity is stronger with respect to the claims contained in Groups 1 and 3 of the 082 Patent, i.e., those that pertain to the distance between crossties and breaks in a rail. Both of these phenomena are plainly visible when viewed from above, and there is a basis for asserting that the challenges they present, and the use of automated inspection tools as solutions, are both recognized in the prior art. However, none of the prior art describes systems or methods that are comparable to those found in the 082 and 249 Patents.
[149] An ex post facto analysis of an invention is potentially unsound (The King v Uhlemann Optical Company (1949), 11 CPR 26 at 46). An allegation of obviousness may be weakened if the evidence does not explain directly, or by inference, why the claimed invention was not discovered by others (Apotex Inc v Bayer AG, 2007 FCA 243 at para 25). Tetra’s challenge to the validity of the 082 and 249 Patents entails breaking the inventions down into their constituent parts, demonstrating that the parts were individually known, and then asserting that the combination was obvious. The Federal Court of Appeal warned against precisely this kind of analysis in Bridgeview Manufacturing Inc v 931409 Alberta Ltd (Central Alberta Hay Centre), 2010 FCA 188 at paragraph 51.
[150] I therefore conclude that the claims of the 082 Patent and the 249 Patent are not invalid on the ground of obviousness. The identification of the particular problems, and the use of machine vision and specified calculations as possible solutions, required invention and was not obvious as of the priority dates.
[151] Section 42 of the Patent Act grants the patent holder the exclusive right, privilege and liberty of making, constructing and using the invention and selling it to others to be used. A patent is infringed by any act that interferes with the patentee’s full enjoyment of the monopoly granted (Monsanto Canada Inc v Schmeiser, 2004 SCC 34 at para 34 [Monsanto]).
[152] Pursuant to s 55(1) of the Patent Act, any person who infringes a patent is liable for all damages sustained by the patentee after the grant of the patent by reason of infringement. Infringement is determined by comparing the products that are said to infringe the patent with the patent’s claims as construed by the Court. If the 3DTAS contains each of the essential elements of the claims in issue, then Georgetown’s patents have been infringed.
[153] The burden of proving infringement rests with the party that alleges it (Monsanto at para 29). The burden therefore falls upon Georgetown.
[154] Georgetown submits that Tetra and CN jointly infringed the 082 and 249 Patents by common design: CN installed the equipment provided by Tetra on its rail cars in order to collect image data and perform initial processing; Tetra then conducted further processing and analysis of the data collected by CN.
[155] In Packers Plus Energy Services Inc v Essential Energy Services Ltd, 2017 FC 1111 [Packers], Justice James O’Reilly found that a patented method for wellbore fluid treatment was not infringed because the defendant had not performed all of the steps of the method described in the patent. Although the defendant supplied and installed the necessary equipment, third party companies independently performed other essential steps. Justice O’Reilly held that “parties who act in concert to commit a tortious act can each be found liable if all of the parties involved arrived at an agreement to carry out the tort”
(Packers at para 48). However, in that case, no agreements had been entered into. The named defendant was therefore found not to have infringed the plaintiff’s patent. The decision has been appealed.
[156] Packers concerned a patented method. In the present proceeding, Tetra is alleged to have supplied an infringing system to CN. With respect to the method claims, Georgetown notes that Tetra and CN entered into an Equipment Agreement, a Licence Agreement and a Service Agreement. Assuming, without deciding, that Tetra’s provision of equipment to CN and its subsequent processing may be insufficient to support an allegation of infringement of the method claims of the 082 and 249 Patents, in my view the three written agreements satisfy the criteria of Packers for the commission of the impugned acts in concert.
[157] In any event, Tetra does not say that some of the essential steps of the methods described in the 082 and 249 Patents were performed by CN, and that it should escape liability as a result. Nor does Tetra dispute that it entered into agreements with CN to achieve a common objective. It is therefore unnecessary to further consider Georgetown’s argument respecting infringement by common design.
[158] The parties agree that all elements of the claims in issue are essential. There is no dispute that many of the elements of the claims of the 082 and 249 Patents are also present in the 3DTAS. However, Tetra denies that the 3DTAS exhibit the following five features:
•analyzes “a frame”, “at least one image”, and/or “a frame of the plurality of images”;
•determines crosstie and tie plate contour;
•uses an ROI;
•compares the orientations of crosstie and tie plate contours; and
•determines an actual delta between the rail base and the crosstie.
[159] According to Georgetown, the 3DTAS creates an elevation map by capturing data contained in individual frames, and then analyzing the data from many frames. This occurs as follows: two SICK Ranger cameras are mounted on the 3DTAS. They alternate in taking pictures of a rectangular region around a laser line that is draped across the surface of the track. Each image, recorded as pixels, is saved for a certain amount of time inside the SICK Ranger camera. The 3DTAS simplifies the image by removing the foreground and background of the railroad track. The simplified digital picture is then converted by the SICK Ranger camera into what is called an “elevation profile vector”
, which is a form of image data. Each digital picture is converted into a unique vector, which retains the height information of the line position. The SICK Ranger camera assembles the vectors into blocks of 5,000, which are stored in matrices on a hard drive. The data are eventually uploaded and sent to Tetra, where post-processing occurs through de-multiplexing and sequencing of the elevation data vectors. A 3D map is then created.
[160] Tetra responds that the 3DTAS does not analyze 2D images sequentially, as contemplated by the 082 and 249 Patents. Instead, the 3DTAS uses elevation maps. These are described by Tetra as “matrices of elevation values, constructed from 3D elevation scan vectors”
, which may be presented graphically. Tetra says that the elevation maps are not processed as images in the manner described by the Patents, and no analysis is done by the 3DTAS on 2D frames captured by the camera.
[161] Mr. Parent insisted that the systems described in the 082 and 249 Patents do not analyze images that have been combined or processed to create a 3D elevation map, but only 2D images captured by the optical sensors. Mr. Parent said that the algorithms in the 082 Patent always refer to 2D images, and these algorithms cannot be applied to a 3D elevation map.
[162] However, Mr. Parent also acknowledged that a digital camera captures images in a manner similar to that of a human retina. The image genuinely exists within the camera. The image data that are subsequently processed may not be presented in this manner, although they can be. Algorithms permit images to be presented to the human eye in a variety of ways. It is not necessary for the camera to be capable of digital image processing. A standard camera may be used.
[163] Mr. Parent compared the creation of a 3D elevation map to re-assembling the slices of a loaf of bread, then viewing the uppermost crust. In cross-examination, he agreed that 2D images are preserved in a 3D elevation map, which comprises a sequential assembly of height profiles obtained from the camera.
[164] Dr. Mesher agreed that an elevation map is a collection of data. He explained that in each line of the elevation map, an individual elevation vector is displayed which portrays elevation as a shade: white is near, dark is distant, and grey is somewhere in between. A frame is a collection or a data structure of pixels, and the 3DTAS analyzes a compilation of individual elevation vectors.
[165] Georgetown points to paragraph 0044 of the 082 Patent, which states: “[a]s can be seen in Figures 11-12, the compiled image data forms a three-dimensional representation (X, Y, and Z) of the area of the track bed”
. In other words, the patent specifically contemplates the creation of a 3D representation from data compiled from 2D images captured by the camera.
[166] I am therefore satisfied that the 3DTAS, when it creates an elevation map, necessarily analyzes “a frame”, “at least one image”, and “a frame of the plurality of images”. A 3D elevation map is simply a construct of the data derived from multiple 2D images displayed sequentially.
[167] Georgetown says that the 3DTAS also satisfies the essential element of determining crosstie and tie plate contour. In a 2D image, a contour is displayed as a single line. Conversely, in a 3D elevation map, a contour is displayed as a continuous surface, which is generated by compiling height information from multiple 2D images or elevation vectors. Mr. Parent agreed with this interpretation in cross-examination.
[168] Nevertheless, Tetra argues that the 3DTAS collects vectors of unconnected elevation points. Tetra says that in the 3DTAS, the mean or median elevations of the plate and tie regions represent two single floating point numbers, from which no contours can be calculated.
[169] I prefer Georgetown’s explanation. I am sceptical of Mr. Parent’s statement that the 3DTAS “collects vectors of unconnected elevation points”
. If the vectors are truly unconnected, it is unclear how they could be combined to produce a 3D elevation map.
[170] Tetra’s position here is similar to the one it has taken regarding whether the 3DTAS analyzes a frame of the plurality of images. Tetra maintains that the 3DTAS does not analyze 2D images at all, but instead analyzes elevation maps. In my view, this is a distinction without a difference. An elevation map is created from 2D images captured by optical sensors. A contour remains the same regardless of whether it is displayed singly as a 2D image, or sequentially as a 3D elevation map. The only difference is that in a 2D image, a contour is displayed as an outline, while in a 3D elevation map, it is displayed as a surface.
[171] Georgetown asserts that the 3DTAS uses ROIs to determine and compare contours, and that the ROI is the surface of the tie plate and the adjoining area of the tie. According to Dr. Myler, in order to obtain the contour data for tie plates and crossties, the 3DTAS system first locates the tie plate holes. For wooden ties, it searches its library for tie plates with matching hole patterns. Once the model is identified, the known dimensions of the model are used to define a bounding box for the tie plate. The 3DTAS system then defines the ROI as the areas at the lateral edges of the tie plate, and the corresponding region on the nearby tie.
[172] Dr. Mesher was shown a drawing of a tie plate adjacent to a tie. He was asked to indicate the area that is used by the 3DTAS to calculate the tie plate surface height. He did so, but declined to describe this as an ROI within the meaning of the 082 and 249 Patents. Instead, he opted for a more generic description: “a region that’s interesting”
.
[173] Tetra argues that Georgetown has conflated the ROI with the bounding box. Dr. Mesher explained that an ROI is an area to which a view is restricted in order to find something and improve processing. However, he emphasized that no processing is done by the 3DTAS to locate an ROI. Instead, once the bounding box is defined, the ROI is determined mathematically without needing to search for anything else.
[174] Tetra says that, in the contexts of the 082 and 249 Patents, ROIs are used to highlight horizontal and vertical zones in order to detect specific features. Conversely, the 3DTAS system locates tie plate features by processing the entire elevation map to locate spikes and holes, and then selecting the best known plate using spike and hole pattern matching. Once defined, the plate bounding box is used to define other areas that are required to perform elevation calculations. However, the calculations that determine differences in elevation are based on plate and fastener locations, and require no knowledge of the bounding box. Tetra therefore concludes that the 3DTAS does not use an ROI to restrict the search for any feature in the 3D elevation map, either vertically or spatially.
[175] Insofar as it is possible to follow Tetra’s argument, it appears to be premised on the assumption that the 3DTAS does not analyze 2D images at all, but instead analyzes elevation maps. I have rejected this distinction elsewhere, and for similar reasons, I reject Tetra’s assertion that the 3DTAS processes “the entire elevation map”
rather than analysing a specific ROI. I accept Georgetown’s evidence and argument that the 3DTAS makes use of an ROI in the manner contemplated by the 082 and 249 Patents.
[176] The ordinary meaning of the word “orientation”
is the relative position of something. In Dr. Mesher’s cross examination, he agreed that the 3DTAS calculates plate cut values by subtracting the height of the tie and the thickness of the tie plate from the height of the tie plate. The thickness of the tie plate is found in the library of tie plate models. Because this procedure involves comparing the relative positions of tie plates and adjacent ties, Georgetown maintains that the 3DTAS compares the orientations of their respective contours.
[177] As noted above, Tetra insists that the 3DTAS does not determine crosstie and tie plate contours. If this is true, then it follows that the 3DTAS does not compare their respective elevations. Dr. Mesher testified that comparing the orientations of crosstie and tie plate contours requires a comparison of the angle between the contour of the tie plate and the contour of the crosstie. He expressed the view that this cannot be done with a single measurement point, which is what the 3DTAS uses in its calculations. A single measurement point is not a contour.
[178] According to Tetra, the 3DTAS system measures an average height for one region at the tip of the plate, and an average height for an adjacent region on the tie. The difference between the two averages is reported. While Tetra calls this measure “plate cut”
, it is not the real value of the plate cut because the height of the plate is only an estimate.
[179] I have concluded elsewhere that the 3DTAS does in fact determine crosstie and tie plate contours, although it uses an algorithm to display them as a continuous surface rather than as a line profile. Tetra admits that the 3DTAS measures the height of the region at the tip of the plate, and the height of the adjacent region on the tie, then calculates the difference. In my view, this amounts to comparing the orientations of crosstie and tie plate contours.
[180] The essential element of determining an actual delta between the rail base and the crosstie is found only in the 249 Patent. The actual delta is simply the actual difference in height between two track components. Dr. Mesher agreed with Dr. Myler that the 3DTAS determines the difference in height between the rail base and the adjacent crosstie in order to calculate the thickness of the polymer pad. It calculates the average or mean of two points on the rail base, and then subtracts the average or mean of the heights at three points on the tie. It then further subtracts the thickness of the rail base.
[181] There is no serious dispute between the parties that the 3DTAS determines the difference, or the “actual delta”, between the rail base and the crosstie, although Tetra notes that the measurement reported by the 3DTAS is not precise. However, the formulas for determining the actual delta provided in the 249 Patent include a TC factor to account for the lean or tilt of the truck, whether left or right. Tetra therefore argues that the calculation of “actual delta” necessarily requires a consideration of tilt.
[182] Georgetown acknowledges that the 249 Patent includes formulas for determining rail seat abrasion as preferred embodiments of the invention. However, Georgetown says that the claims should not be limited to only what is disclosed in the examples (citing Dableh v Ontario Hydro (1996), 68 CPR (3d) 129 at 144 (FCA)). Furthermore, the claims of the 249 Patent encompass the detection of rail seat abrasion even when the track is level. If there is no vehicle tilt, then no correction is required and the TC value in the formula is 0.
[183] Even if TC were found to be an essential element of the claims in issue, Georgetown says that the 3DTAS does in fact correct for tilt. As the elevation data vectors are converted into a 3D elevation map, a “real world elevation correction”
is performed. Pixel units are converted into engineering measurements, such as millimetres or inches. In order to obtain real world units of measurements, the 3DTAS makes adjustments for vehicle dynamics that affect the distance between the surface and the cameras. Dr. Mesher did not deny that the 3DTAS corrects for vehicle dynamics, but said that it does so only in the longitudinal dimension. Georgetown says this is sufficient to establish that the 3DTAS corrects for tilt.
[184] According to Tetra, the 3DTAS applies “Dynamic Vehicle Motion Corrections”
. This process is intended to counteract the effects of vehicle suspension bounce in the direction of travel. However, Tetra asserts that it does not address the height differences between the left and right rail heads. According to Tetra, there is no need for the 3DTAS to make this adjustment, because the minor difference in rail height caused by any transverse tilt of the cameras does not affect the degree of precision required by its customer.
[185] In the discussion of Claims Construction, above, I have found that the 249 Patent does not contemplate that compensation for tilt will always be necessary as the system moves along the track. Instead, the system compensates for tilt only when this phenomenon is “encountered”
.
[186] I therefore conclude that the 3DTAS determines the difference, or the “actual delta”, between the rail base and the crosstie in the manner described in the 249 Patent. The algorithm for determining the actual delta disclosed in the patent does not require a consideration of TC in all circumstances. The claims of the 249 Patent encompass the detection of rail seat abrasion even when the track is level.
[187] For the foregoing reasons, I find that the 082 and 249 Patents are valid. The essential elements of the asserted claims are also found in the 3DTAS. Tetra’s sale of the 3DTAS to CN and its support of the system therefore infringed both Patents.
Email this Content