Fast Search Features in SharePoint 2013 for better document search and increasing productivity

Microsoft acquisition of Fast Search for $1.2Billion

In the year 2008, Microsoft acquired a technology company called Fast Search & Transfer ASA for a whopping US$ 1.2 billion. With this acquisition, Microsoft capitalized on the reputation Fast Search had built for itself in the field of enterprise search and tightened its grip on the workspace collaboration ecosystem by introducing the Fast Search feature in SharePoint 2013. Initially launched as a separate component, Microsoft has since invested considerable effort, time and money to integrate “FAST solution” into SharePoint. And the integration is complete in the latest version of SharePoint 2013 where the best components of Search Server Express, SharePoint Enterprise Search and FAST Search Server are brought into a single SharePoint Search platform.

Save Time Using SharePoint 2013 Search Capabilities

While earlier versions of SharePoint facilitated content search via content web query, it had its limitation when the searches were out of Site collections. Now SharePoint 2013 can be used to index content from a wide variety of sources (websites, file shares, exchange folders, lotus notes, etc.) using third party connectors. SharePoint 2013 builds a single repository of index which allows users to provide a single enterprise search center experience and avoid the need for going to different applications to search for the same type of information.

Fundamental architectural changes help overcome the earlier version’s limitations and the user can now index content from multiple site collections thereby fortifying SharePoint’s existing WCM capabilities. These improvements along with new features such as query syntax, query suggestions and results preview pane (hover for full content previews) helps users be more productive by getting faster and more precise search results.

Precise and Centralized Search Results

By leveraging Fast Search content capabilities and fusing it with SharePoint’s native search features, Microsoft SharePoint 2013 now renders a centralized dynamic interface. These interfaces are easy to navigate and have a more appealing social user interface making User Experience more productive and engaging. Inbuilt content and usage analytics processing components bring better relevance to search results.

Query suggestions in these features make it much easier to use SharePoint search functionality. These query suggestions offer smart guessing and intuitive assistance that saves time by rendering right search results and assists in preventing typos. If enterprises wish to extend search capabilities to anonymous users, SharePoint 2013 allows integration with Mavention Query Suggestions for public facing SharePoint websites.

Search Latest Documents as Soon as they are Added

In SharePoint 2010 we had to schedule crawls to index the latest content sets being added on a continuous basis. In case the content sets were huge and exceeded the crawl slot, the next scheduled crawl had to wait for the previous crawl to finish indexing the content set. Due to this time lapse the indexing process would become involved and time consuming.

To ensure that search results are updated at the same rate as that of content addition in an enterprises’s SharePoint site, Microsoft has introduced a continuous crawl feature with SharePoint 2013. It allows multiple incremental indexing jobs to process paralleling ensuring that no content is missing from the search results.

In order to keep the content sources and search index fresh, SharePoint 2013 facilitates continuous and automatic crawling which eliminate the need to schedule incremental crawls and authentication by allowing administrators to configure crawl schedules. SharePoint 2013 also enables users to remove items from the search index by using crawl logs.

SharePoint 2013 Integration with Office Web Apps

SharePoint 2013 offers smart documents and web pages’ previews and eliminates the redundant task of surfing through files to find the correct one. SharePoint 2013 smart document viewing enables users to get a preview of any document or file by simply resting the cursor on them. This saves a lot of time and helps users become more productive.

SharePoint 2013 allows smart integration with the Office Web Apps both for viewing and editing office content and search previews of documents. Users can choose a document from SharePoint 2013 library or SkyDrive, and edit documents with all Microsoft office functionality or on supported mobile applications. Office web apps, when integrated with SharePoint 2013, render hosts of other features like co-authoring, change tracking, quick preview and more.

With these enhanced search capabilities, SharePoint 2013 can help enterprises with best in class and high end search practices and help their workforce become more productive.

Read Full Article and about the authors on http://www.cmswire.com/cms/social-business/sharepoint-2013-fast-search-features-drive-enterprise-productivity-020703.php

EXCEL vs BI tools – Towards a Data Driven Culture!

The Great BI vs Excel Debate

As Business intelligence vendors slog it out to flex their muscles on enterprise stage and has experts talking lengths on BI, one thing that is often dropped out is the real protein aka Excel; from where the BI concept got ripped and is now flaunting its curves on enterprise’s Dashboards.
One cannot undermine the fact that, the DNA of “business analytics” which is the core part of business intelligence still remains deeply entrenched in analytical capabilities of Excel (that has long been and still is the mainstay of enterprises’ analytics needs). However, at present, the real power of any tool or application is evaluated on the basis of its ease of use and intuitive features that can help even non tech savvy people get relevant insights as per their needs
without relying too much on IT staff.

The Great BI vs Excel Debate

So, let’s uncover which areas, Excel needs to flex its muscles in its run up to the Business intelligence race.

Types of Users and Familiarity

Proponents of Excel are those power users who can perform almost all Excel analytical stunts. But, when it comes to less tech savvy frontline executives or other personnel, mastering Excel would be a tough pursuit. Here’s where modern BI tools makes it easy for even the non-techie employees perform slicing and dicing, do some data mash-ups with intuitive visualization features.

Timely re-configuration and processing

One of the areas where BI tools scores over Excel is providing real time data which is outside the purview of Excel. The ability to connect directly to the databases and heavy under the hood plumbing makes real time data monitoring easy and actionable. Updating and re-configuring data also consumes a lot of time in Excel and sometimes runs out of memory as well. Due to these inconsistencies working with Excel becomes difficult. The amount of time taken to download data in Excels sheets and tons of manipulation done in order to get relevant insights from disparate applications seems quite a daunting task.

Visualization and Web access

Modern BI tools provide rich interactive visualization, web access to analytics as well as new forms of interactive data visualization which still remains a far cry for Excel.

Error Probability

Mismanagement of data, corrupt files and inadequate security makes Excel more vulnerable to errors. Even, power users who can build their own macros might still run the risk of making hidden errors. Here, BI tools that are integrated with the databases help in master data management and these tools being rigorously tested, come as a saving grace for many users.

Complex Decision Making

When it comes to complex decision making that requires manager to access information from various applications like SAP, CRM, ACCOUNTING etc. and then downloading them to Excel sheets and performing analysis becomes a complex procedure. On the other BI tools are far more sophisticated and can be easily integrated with cross functional applications to provide meaningful insights without banging heads on the Excel walls.

Complex Marketing Research and statistical analysis

In order to perform complex research and statistical analysis, Excels fails to leave a mark as compared to the like of Spss and Sas. These softwares provide domain driven data mash up capabilities and are far easy to use then Excel. Further, managing unstructured data becomes difficult in Excel.

Expense

BI tools are more expensive then Excel, this is one area where Excel holds sway over BI tools. For smaller organization Excel can help solve much of these BI challenges in cost effective way, when it comes to big organization, Excel doesn’t stand a chance against mighty BI providers like COGNOS, QLIKVIEW, TABLEAUE and the like.

Size of data

Though Microsoft has made significant changes in Excel to enable power users to manage large amount of data, Excel still falls short of space while handling large amount of data. This is one major problem with Excel comparatively with BI tools that come with high storage capacity to handle large data sets.

Computed measures KPIs, Etc.

Though Excel can perform complex analysis, when it comes to performing analysis to measure KPIs using disparate data sources, BI tools are miles ahead.

Compliance

BI tools offer better compliance management capabilities than Excel. BI tools provide much more reliable data for auditing various standards where quantification of process and information flow is reliable and streamlined.

Addressing time difference issue in Onsite / Offshore development model

Having handled complex Java projects at our offshore development center at Trigent’s Bangalore Office, here’s my take on the oft-discussed topic on time difference issue attached with onsite/offshore development models. For sake of readers convenience, I have also included some key collaborative steps that can be put into practice for making smooth progress towards meeting timelines.

Addressing Time Zone Difference with Offshore Software Development Partners

Below are some of the steps to be taken to make sure collaborative development work is smooth & successful in an onsite / offshore product development:

  1. Know your team – This is the most important first step in any collaborative development. This can be achieved in many ways.
    • Have onsite team members visit offshore facility or vice versa
    • Have video conferencing calls
    • Share photos / pictures of the team members
  2. Have daily meetings – Meetings with onsite team daily will make the team understand what each member is doing
  3. Collaborative design – Offshore team work together with onsite team members in designing the application. To beat the time difference the offshore team can work extended hours during design phase
  4. Pair programming – This can be achieved by having an offshore team member & an onsite team member both work on the same module.
  5. Peer review – Have offshore team member’s code reviewed by on-site team member & vice versa. This can help in achieving on-site team’s expectations
  6. Continuous build & integration – Have a CI system in place which will make sure that clean code is checked-in at any time.  The following steps can be implemented:
    • Automatic build will happen whenever code is checked-in
    • The CI system executes the regression test suites to make sure nothing breaks
    • Notify the team on any failures so that team can act on it
    • Make sure the team runs the complete build before their EOD. This will ensure that the onsite team gets clean code to work on.
  7. Regression test suites – Build regression test cases & integrate it with continuous build. This will make sure that the checked-in code has not broken somebody else code.
  8. Disciplined code check-in – The following can be enforced:
    • Have visibility into what each member is working on, which files etc.
    • Make sure that one does not override somebody else code.
    • Always see the repository changes before check-in.
    • Periodically check-in smaller logical changes
    • Don’t check-in partially completed code
  9. Communicate – This is an important step. Following options can be implemented for effective communication:
    • Create a group e-mail account which will include on-site team members
    • Use Skype or other IMs and communicate daily
    • Exchange each other’s alternate communication details like phone, mobile etc.
    • Have contact details of all the team members in common central place & accessible to all.
    • Identify offshore team member who can be contacted during emergency

Operational BI at Banks

Embedding Operational Business Intelligence (BI) tools to several banking operations can help operational managers get actionable insights on operational bottleneck, historical data etc. for managing, monitoring and for controlling contingencies.

By getting actionable and real time  insights on operational bottlenecks, managers can take corrective measures to eliminate operational flaws.  Let’s consider a simple example of a cheque clearance process at banks and see how Operational BI can help you get valuable insights to improve functional inefficiencies.

Operation BI for Banking and financial operations

‘A’ deposits a cheque in favor of ‘B’s account on day one. Afterwards, B’s bank processes the cheque which includes validation of cheque’s credentials and passes it on to A’s Bank. The cheque validation processes include a series of steps to identify instances of frauds, validation of  the sufficient amount of money at A’s account,  signatures,credentials, etc. and other issues that may obstruct the fund’s transfer process. The settlement process is over when A’s bank clears the payment to B’s Bank.

The cheque clearance process might seem simple at first sight, but there are several intricacies involved within the process. Firstly, there are many instances of frauds and tamperings such as counterfeit, forgery or alterations. Secondly, there are unpaid instances such as insufficient funds at customers account or there are several fields wrongly entered such as dates, wrong signature etc. It would be far too simple to manage these cases if there were less number of transactions, but consider how difficult it would be if there are millions of cheques being deposited and cleared every day. Besides, it is difficult to give answers to customers who inquire about why their cheque’s being bounced or other queries related to their settlement processes.

Where does Operational BI fits in?

Since the settlement process is complex and takes a lot of time in terms of the number of days to clear a cheque, wouldn’t it be nice if fraud cases could be detected real time to reduce operational risks?  Well, here’s a tip on how you can do this? Business rules can be applied by analyzing history of frauds that would signify potential fraud while managing daily operations and reducing the number of frauds. Operation BI can also help customers get real-time information about their queries thereby reducing user reaction for any issue. It can also be integrated with functional processes and functional applications to get actionable insights. In the cheque clearing process, an operational BI can help managers know how many cases of frauds appeared per day, cases of insufficient amount, cases of halt, or number of cheques that were forged or tampered.

Interesting Video by Logi Analytics themed around The BI Chocolate Cake Problem”.”

Here is a very interesting video by LOGI analytics. The video metaphorically describes enterprises’ end-users as kids and toddlers demanding for variety of chocolate cakes (figuratively representing data reports) and their increasing reliance on IT staff to get personalized reports on time. It also features IT staff’s pathos on how they are flooded with the demands for variety of cakes(aka reports) and their inability to provide reports on time apart from focusing on their workday routine jobs.

So watch out for yourself before I spill the beans in this piece.

Click on the Image To Play

‘The BI Chocolate Cake Problem’

Is the Cloud ready to “Birst”?

Yes! You heard it right it is “BIRST”. You might have come across it, If you are one among those who are evaluating the current Business Intelligence (BI) Market, or have plans to deploy BI solutions in Cloud and so on and so forth.

Challengers in Gartner Magic Quadrant – Logi Analytics vs Birst

BIRST vs LogiAnalytics – Challengers in Gartner Magic Quadrant in BI

BIRST is a business intelligence and analytic service provider based out of San Francisco which is steadily gaining grounds in cloud platform. Birst also featured as a CHALLENGER in “2013 Gartner’s Magic Quadrant for Business Intelligence and Analytics”. Another contender in Gartner’s Challenger quadrant is LOGI Analytics (LogiXML) riding high on its ability to rapidly create, business intelligence and analytics application on-premise. These are two companies to watch out for in coming days as they seem to get much of attention in BI space and both are well positioned in cloud and on-premise platform respectively.

Speaking of LOGI Analytics, a relatively mid-size company as compared to other major BI players it packs a punch when it comes to its ease of use, mobile capabilities, multiple web browser support and unlimited user capabilities. It has a dominant market share in reporting OEM market as it is highly embeddable, along with other open source players like Pentaho and Jaspersoft. The LOGI V11 saw significant acceptance among enterprises in 2012 with enhanced visual presentation and interactive functions, such as advanced selection, filtering and functions (according to Gartner). BIRST is flexing its BI muscles to get a share of this market.

What’s the Future?

When it comes to Cloud, BIRST is gaining momentum given the breadth of functionality and features besides its low cost of ownership. Another competitor who is faring well on cloud platform is GOOD-Data. BIRST also faces stiff competition from other cloud based vendors who have a solid grip in Mobile space.

The Cloud vs On-Premise debate is not going to end anytime soon.Yet players in both the categories are expanding their market shares. Massive developments are being made to offset cloud vulnerabilities involving security, data integrity and privacy. From data discovery phase to diagnostic analytics phase, BI is still growing. Those fast enough to provide predictive and prescriptive solutions to the market as early as possible will dominate. Pro tempore, it is interesting to watch how the industry is providing diagnostic analytics solutions to enterprises. In addition, as we are expanding our network of proprietary BI vendors, we recently partnered with LOGI-Analytics to help enterprises deploy the best of breed analytics solutions. Read more about our partners

Handling CSV files Using HSQL Text Tables

As programmers, we all know what a CSV file is all about. For novice, a CSV file is a “Comma Separated Values” files having some kind of data. Each line represents a record while each value is separated by a comma “,”.

Let’s take a simple example:

Say the CSV file contains order data. Each order record contains data about a particular order like order number, product code, quantity, ordered by, date etc. Now lets take a hypothetical requirement which says, the user needs to know how many orders were ordered per product.

Well given this requirement, as a programmer what are our options?
Option 1]  At the basic level, read the order file, parse each line and write the logic to get count of orders per product.
Option 2] Load this CSV file into a database like MySQL and write database queries to get the orders per product.
Option 1 is good if the file size is small and when we have to deal with less number of orders. If the file size is big then we will get into other issues like performance, memory etc.

Option 2 is good but it requires database administrative capabilities to load the data into database. Of course we can use in-memory database but still we may need to use database utilities to load the files into the database. This will lead to duplication of data that is data in CSV file as well as same data loaded to database table.

So where does this leave us? Have we run out of options? I am sure we would have tried & used the above two options . But I wanted a different approach. Following questions were lingering in my mind:
1] Why can’t I write SQL queries against the CSV file itself. After all it’s like any RDBMS table.
2] Why should I load the CSV file into some database before I query it?
3] Why can’t I create a database table & attach the CSV file to it?
Well my quest for the answers led me to “HSQL Text tables”. HSQL is a database which can run in-memory. No pre-installation, configuration is required to use this database. It has a feature to create a “Text” table and attach the CSV file as a source to that table. No need of manually loading the data into this table.

So for our requirement, we used HSQL Text tables. Below are the steps to use Text tables:
1] Set the system property:
System.setProperty(“textdb.allow_full_path”, “true”);
This will enable HSQL to load the CSV file from any path. In our case, we wanted to load the file from network path.
2] Create Text table using the below command:
“CREATE TEXT TABLE tbl_Order ord_no varchar(30), prod_code varchar(30)…..”;
3] Create any indexes, if required.
4] Attach the CSV file as a source to the table using the below command:
SET TABLE tbl_Order SOURCE “x:\server\order.csv”;

Voila. That’s it. Once the above steps are done, we can execute SQL queries as if its a normal table.

In a nut shell, HSQL Text tables are powerful & easy to use. It eliminated the lower level of file reading & parsing the file as well duplication of data and still met the requirement.

SharePoint Development – What it takes to be the best SharePoint Developer?

SharePoint Developer – What is expected of him today?

A SharePoint developer’s role goes well beyond merely customizing the platform as per client’s needs. They should possess the acumen to foresee business value. Put it this way, what is SharePoint ? ..It’s a collaboration platform.  That’s anybody’s guess. The real value of SharePoint comes when the term collaboration converts into measurable business value and help improve bottom-line.

When a SharePoint developer cultivates an acumen to foresee “collaboration benefits” converting into real business value, that’s where he earns extra brownie points. Now, let’s delve deeper into various aspects of SharePoint development and skillset of a SharePoint developer that goes into implementing the best-in-class SharePoint solutions.  Enterprises have different opinions and views when it comes to assessing the skills required to become a SharePoint developer.

SharePoint developer roles

SharePoint Developers Roles and Responsibilities

When the development requirements are clear from clients end, then the task of SharePoint developers becomes pretty much easy.  However, when a client requires an end-to-end SharePoint consultancy, the task of a SharePoint developer calls for more than just out-of-the-box customizing or regressive custom coding. This is because when it comes to developing a complex platform like SharePoint, there are many dimensions involved. Some of them are .net development skills, business analytical skills, intranet portal designing, information architecture designing and so on and so forth.

SharePoint is widely used across industries. A little knowledge of its application areas across industries can give  developers an upper-hand for more reasons than one. For one, developers can be aware of specimens, procedures and client’s requirements that are applicable to a particular segment.

Developing a complex unified collaborative platform like SharePoint entails more than programming skills to meet client’s requirement. SharePoint development involves more than merely having programming skills associated with the SharePoint framework like.Net, HTML etc. It also encompasses deeper understanding of business implications of deploying a SharePoint solution to customizing them to meet end user or business needs. For a small SharePoint project, a developer’s role may include all of the above. However, when it comes to a big enterprise wide deployment; it takes a lot more than that.

Here are some of the technical skills of a seasoned SharePoint Developer:
1. SharePoint Developer must possess excellent Communication Skills
Sounds kind of obvious, but this one is often dropped out. Especially when a SharePoint developer is working in some offshore countries such as India, Ukraine, Philippines etc. for that matter. These skills not only help build strong rapport with clients but also helps in understanding complex business requirements and scenarios.

2. Programming Skills
SharePoint is built over .Net framework and most of the customization and development scope involves programming languages like.Net, HTML, JavaScript,Jquery, etc. Apart from these, a SharePoint developer must possess associated technical knowledge of admin and servers associated with SharePoint like SQL Server administration, Windows Server Administration, active directory control, etc.

3. Developing Web-parts to ease end users need
Web parts are elementary units of SharePoint web pages and are the essential features of SharePoint. Using these web parts, users can easily create, modify and transform web pages. Hence, ability to create custom web-parts that can solve a specific end-user problem, requires mastery over .Net programming skills.

4. Knowledge of Industry Specific Processes
The purpose of SharePoint is to make workforce more productive by rendering collaborative tools that can help them become efficient in managing their day to day tasks. While SharePoint is widely known as a “Collaboration Platform“, knowing how its features can help end users manage their tasks efficiently is also vital.

5. Microsoft Certification helps to an extent
SharePoint is one of the flagship products of Microsoft. In order to hone SharePoint programming and coding skills, Microsoft has designed several certification courses to authenticate SharePoint programmers in order to customize SharePoint according to end users and business requirements. These certifications require developers to demonstrate continued ability to develop SharePoint.

6. Basic Web Designing Skills
SharePoint can be used to build both corporate intranets as well as extranet portals.  The idea is to provide a centralized platform where internal and external stakeholders can communicate and collaborate with each other effectively. Hence, a sound knowledge of web designing can go a long way in developing a SharePoint Site.

Interesting Info-Graphics by Start up Evangelist Anna Vital

Here’s a nice info-graphics where Anna Vital nicely explains how to overcome fear when bitten by entrepreneurial bug. The info-graphic talks about the attitude and the metal mindset needed to drive a start up. It may sound kind of  basic success model, but inside it lies transitional phases that an entrepreneur has to go through. Plus, the pressure to perform at various phases to get several tranches of funding makes it worse for an entrepreneur to keep up the spirit and enthusiasm of  the team.That’s where a simple info-graphics of entrepreneurial wisdom can help entrepreneurs  get moving. Have a nice day!

Java Programming – Automatic form filling in PDF using JAVA codes

Java Development to Control Crucial Business Aspects

In the world of automation, who would like to take the pain of manually filling out redundant form fields? Especially when forms control critical aspects of your business and time challenged professional may find it real hard to cope with it? Besides, things can really get tedious when these forms are to be managed in PDF formats!!

Wouldn’t it be nice if we had a provision to dynamically fill PDF documents by pulling information right from the database? …that would certainly save a lot of manual efforts wouldn’t it?? That’s exactly one of our clients wanted and here’s how I used NITRO PDF and JAVA Code to automate much of the heavy lifting.

Here’s how it works

So let’s get started. Let me give a brief roundup on NITRO Reader. You might have used it to convert your word files to PDF. But there’s more you can do with the trial version to start with. You can actually create form fields in PDF documents. But that’s one part. The real value of this post comes when you combine the JAVA snippets that I have given below to automate the hard way out.

Here is the screenshot which have form fields that have been created in PDF document using NITRO

Java Programming Part

Now let’s get started with the coding part. The JAVA code mentioned below interacts with the database and can automatically pull out relevant information and fill the above form fields.

I have provided the code with comments using some dummy values to make it easily understandable even for a novice java programmer. Here’s how it works.

private ByteArrayOutputStream editPdfDocument() throws Exception {

PdfReader reader = null;

PdfStamper stamper = null;

ByteArrayOutputStream baosPDF = new ByteArrayOutputStream();

HashMap fieldsWithValues = new HashMap();

int user_id = 1234; // unique key for the object

String dirPath = “D: /abc”;/ / directory path

String fileName = “def.pdf”; // name of the file

try {

reader = new PdfReader(dirPath + “/” + fileName);

stamper = new PdfStamper(reader, baosPDF);

AcroFields form = stamper.getAcroFields();

HashMap fields = form.getFields();

Set keys = fields.keySet(); // keys represents the names of all the form fields

fieldsWithValues = fetchFieldValuesForObject(user_id, keys);
// fetchFieldValuesForObject(user_id, keys) method will fetch the values of the fields from Database for object identified by user_id
Iterator itr = keys.iterator();

while (itr.hasNext()) {

String fieldName = (String) itr.next();

String fieldValue = fieldsWithValues.get(fieldName) != null ? (String)(fieldsWithValues.get(fieldName)) : “”;

form.setField(fieldName, fieldValue);

}

stamper.setFormFlattening(true);

} catch (Exception dex) {

if (stamper != null)

stamper.close();

if (reader != null)

reader.close();

throw dex;

}

return baosPDF;

}

The Output – Here you go !!

Automated Filled Form

.Net Programming – Padma Krishnan Speaks!

Senior .NET Project Manager, Mrs. Padma Krishnan speaks about her experience and a .NET project she is currently handling at Trigent. Mrs. Padma Krishnan is an MBA from University of Texas at Austin with specialization in Information Management. She has more than 20 years of experience in Information Technology(IT) and has spent considerable amount of her career serving clients in the U.S. She handles a bunch of .NET projects and helps client get best in class customized solutions that meet their business goals. Continue reading .Net Programming – Padma Krishnan Speaks!